Transcoding system using encoding history information

ABSTRACT

The present invention provides a transcoder having a MPEG decoder and a MPEG encoder, for changing a GOP structure and the bit rate of an encoded bitstream obtained as a result of an encoding process. The MPEG encoder receives a past encoding parameters generated at a past encoding process as a history information, and performs a present encoding process by using the past encoding parameters selectively so that the present encoding process is optimized. Furthermore, the encoder describe the past encoding parameters into the encoded bitstream as the history information so as to reuse the history information in advance encoding process. The picture quality of the video data does not deteriorate even if decoding and encoding processes are carried out repeatedly by the transcoder.

This is a continuation of application Ser. No. 10/342,055, filed Jan.14, 2003, now U.S. Pat. No. 7,469,007 which is a continuation ofapplication Ser. No. 09/265,732, filed Mar. 9, 1999, now U.S. Pat. No.6,560,282, which is entitled to the priority filing dates of Japaneseapplications 10-058118 filed on Mar. 10, 1998 and 10-157243 filed onJun. 5, 1998, the entirety of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

The present invention relates to a transcoding system, a video encodingapparatus, a stream processing system and a video decoding apparatus forchanging a GOP (Group of Pictures) structure and the bit rate of anencoded bitstream obtained as a result of an encoding process based onMPEG (Moving Picture Experts Group) standards.

In recent years, the broadcasting station for producing and broadcastingtelevision programs has been generally using an MPEG technology forcompressing and encoding video data. In particular, the MPEG technologyis becoming a de-facto standard for recording video data onto a tape ora random-accessible recording medium and for transmitting video datathrough a cable or a satellite.

The following is a brief description of typical processing carried outby broadcasting stations up to transmission of a video program producedin the station to each home. First, an encoder employed in a camcorder,(an apparatus integrating a video camera and a VTR into a single body),encodes source video data and records the encoded data onto a magnetictape of the VTR. At that time, the encoder employed in the camcorderencodes the source video data into an encoded bitstream suitable for arecording format of the magnetic tape of the VTR. Typically, the GOPstructure of an MPEG bitstream recorded on the magnetic tape is astructure wherein one GOP is composed of two frames. An example of theGOP structure is a structure comprising a sequence of pictures of thetypes of I-, B-, I-, B-, I-, B- and so on. The bit rate of the MPEGbitstream recorded on the magnetic tape is 18 Mbps.

Then, a central broadcasting station carries out edit processing to editthe video bitstream recorded on the magnetic tape. For this purpose, theGOP structure of the video bitstream recorded on the magnetic tape isconverted into a GOP structure suitable for the edit processing. A GOPstructure suitable for edit processing is a structure wherein one GOP iscomposed of one frame. To be more specific, pictures of a GOP structuresuitable for edit processing are all I-pictures. This is because inorder to carry out edit processing in frame units, the I-picture havingno correlation with other pictures is most suitable. In the actualoperation to convert the GOP structure, the video bitstream recorded onthe magnetic tape is once decoded back into a base-band video data.Then, the base-band video data is re-encoded so as to comprise allI-pictures. By carrying out the decoding and re-encoding processes inthis way, it is possible to generate a bitstream having a GOP structuresuitable for edit processing.

Subsequently, in order to transmit an edited video program obtained as aresult of the edit processing from the central broadcasting station to alocal broadcasting station, it is necessary to change the GOP structureand the bit rate of the bitstream of the edited video program to a GOPstructure and a bit rate that are suitable for the transmission. The GOPstructure suitable for transmission between broadcasting stations is aGOP structure wherein one GOP is composed of 15 frames. An example ofsuch a GOP structure is a structure comprising a sequence of pictures ofthe types of I-, B-, B-, P-, B-, B-, P- and so on. As for the bit ratesuitable for transmission between broadcasting stations, a high bit rateof at least 50 Mbps is desirable since, in general, a dedicated linehaving a high transmission capacity such as an optical fiber isinstalled between broadcasting stations. To put it concretely, thebitstream of a video program completed the edit processing is oncedecoded back into a base-band video data. Then, the base-band video datais re-encoded to result in a GOP structure and a bit rate suitable fortransmission between broadcasting stations as described above.

At the local broadcasting station, the video program received from thecentral broadcasting station is typically subjected to edit processingto insert commercials peculiar to the district where the localbroadcasting station is located. Much like the edit processing carriedout at the central broadcasting station, the bitstream of the videoprogram received from the central broadcasting station is once decodedback into a base-band video data. Then, the base-band video data isencoded so as to comprise all I-pictures. As a result, it is possible togenerate a bitstream having a GOP structure suitable for editprocessing.

Subsequently, in order to transmit the video program completing the editprocessing at the local broadcasting station to each home through acable or a satellite, the GOP structure and the bit rate of thebitstream are converted into respectively a GOP structure and a bit ratethat are suitable for the transmission to each home. A GOP structuresuitable for the transmission to each home is a structure wherein oneGOP is composed of 15 frames. An example of such a GOP structure is astructure comprising a sequence of pictures of the types of I-, B-, B-,P-, B-, B-, P- and so on. A bit rate suitable for transmission to eachhome has a typical value of as low as about 5 Mbps. The bitstream of avideo program often completing the edit processing is decoded back intoa base-band video data. Then, the base-band video data is re-encodedinto a GOP structure and a bit rate suitable for transmission to eachhome.

As is obvious from the above description, a video program transmittedfrom the central broadcasting station to each home is subjected torepetitive decoding and encoding processes for a plurality of timesduring the transmission. In actuality, various kinds of signalprocessing other than the signal processing described above are carriedout at a broadcasting station and the decoding and encoding processesare usually carried out for each kind of signal processing. As a result,the decoding and encoding processes need to be carried out repeatedly.

However, encoding and decoding processes based on MPEG standard are not100 percent reverse processed to each other as is generally known. To bemore specific, base-band video data subjected to an encoding process isnot entirely the same as video data obtained as a result of a decodingprocess carried out in transcoding of the previous generation.Therefore, decoding and encoding processes cause the picture quality todeteriorate. As a result, there is a problem of deterioration of thepicture quality that occurs each time decoding and encoding processesare carried out. In other words, the effects of deterioration, of thepicture quality are accumulated each time decoding and encodingprocesses are repeated.

OBJECTS OF THE INVENTION

It is thus an object of the present invention addressing the problemdescribed above to provide a transcoding system, a video encodingapparatus, a stream processing system and a video decoding apparatuswhich cause no deterioration of the picture quality even if encoding anddecoding processes are carried out repeatedly on a bitstream completingan encoding process based on MPEG standard in order to change the GOPstructure and the bit rate of the bitstream. In addition, it is anobject of the present invention to provide a transcoding system, a videoencoding apparatus, a stream processing system and a video decodingapparatus causing no deterioration of the picture quality even ifencoding and decoding processes are carried out repeatedly.

SUMMARY OF THE INVENTION

In order to attain the above objects, according to the transcoderprovided by the present invention, past encoding parameters generated ina past encoding process are transmitted to the encoder as a historyinformation. The encoder selects the suitable encoding parameters incommensurate with a present encoding process from the historyinformation, the encoder performs the present encoding process by usingthe selected past encoding parameters. As a result, the picture qualitydoes not deteriorate even if decoding and encoding processes are carriedout repeatedly. That is to say, it is possible to lessen the accumulateddeterioration in the quality of picture due to repetition of theencoding process.

According to the transcoder provided by the present invention, theencoder describes the past encoding parameters into the encodedbitstream as a history information so that the history information isavailable in advance encoding process. As a result, the picture qualitydoes not deteriorate even if decoding and encoding processes are carriedout repeatedly at the future.

According to the transcoder provided by the present invention, theencoder describes the past encoding parameters in a user-data areaprovided in said picture layer of the encoded bitstream obtained as aresult of the present encoding process. It is thus possible to decodethe encoded bitstream by means of any existing decoders conforms to MPEGstandard. In addition, it is not necessary to provide a dedicated linefor transmitting encoding parameters generated and used in a pastencoding process. As a result, it is possible to transmit encodingparameters generated and used in a past encoding process by utilizingthe existing data-stream transmission environment.

According to the transcoder provided by the present invention, theencoder selects only a suitable past encoding parameters and describesthe selected past encoding parameters as a variable length stream in thestream. As a result, it is possible to transmit the past encodingparameters generated in the past encoding process without the need tosubstantially increase the bit rate of the output bitstream.

According to the transcoder provided by the present invention, in thepresent encoding process, the encoder selects only optimum encodingparameters from past encoding parameters in accordance with the assignedpicture types of the present encoding process. As a result,deterioration of the quality in picture is by no means accumulated evenif decoding and encoding processes are carried out repeatedly and anoptimum encoding process can be carried out.

According to the transcoder provided by the present invention, adecision as to whether or not to reutilize past encoding parametersgenerated in a past encoding process is made on the basis of picturetypes included in the past encoding parameters. As a result, an optimumencoding process can be carried out.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the invention, reference is made tothe following description and accompanying drawings, in which:

FIG. 1 is an explanatory diagram used for describing the principle of ahigh-efficiency encoding process;

FIG. 2 is an explanatory diagram showing picture types used incompression of picture data;

FIG. 3 is an explanatory diagram showing picture types used incompression of picture data;

FIG. 4 is an explanatory diagram used for describing the principle of aprocess of encoding a moving-picture video signal;

FIG. 5 is a block diagram showing the configuration of an apparatus usedfor encoding and decoding a moving-picture video signal;

FIGS. 6A to 6C are explanatory diagrams used for describing formatconversion;

FIG. 7 is a block diagram showing the configuration of an encoder 18employed in the apparatus shown in FIG. 5;

FIG. 8 is an explanatory diagram used for describing the operation of aprediction-mode switching circuit 52 employed in the encoder 18 shown inFIG. 7;

FIG. 9 is an explanatory diagram used for describing the operation of aprediction-mode switching circuit 52 employed in the encoder 18 shown inFIG. 7;

FIG. 10 is an explanatory diagram used for describing the operation of aprediction-mode switching circuit 52 employed in the encoder 18 shown inFIG. 7;

FIG. 11 is an explanatory diagram used for describing the operation of aprediction-mode switching circuit 52 employed in the encoder 18 shown inFIG. 7;

FIG. 12 is a block diagram showing the configuration of a decoder 31employed in the apparatus shown in FIG. 5;

FIG. 13 is an explanatory diagram used for describing SNR control basedon picture types;

FIG. 14 is a block diagram showing the configuration of a transcoder 101provided by the present invention;

FIG. 15 is a block diagram showing a more detailed configuration of thetranscoder 101 shown in FIG. 14;

FIG. 16 is a block diagram showing the configuration of a decoder 111employed in a decoding apparatus 102 of the transcoder 101 shown in FIG.14;

FIG. 17 is an explanatory diagram showing pixels of a macroblock;

FIG. 18 is an explanatory diagram showing areas for recording encodingparameters;

FIG. 19 is a block diagram showing the configuration of an encoder 121employed in an encoding apparatus 106 of the transcoder 101 shown inFIG. 14;

FIG. 20 is a block diagram showing a typical configuration of a historyformatter 211 employed in the transcoder 101 shown in FIG. 15;

FIG. 21 is a block diagram showing a typical configuration of a historydecoder 203 employed in the transcoder 101 shown in FIG. 15;

FIG. 22 is a block diagram showing a typical configuration of aconverter 212 employed in the transcoder 101 shown in FIG. 15;

FIG. 23 is a block diagram showing a typical configuration of a stuffcircuit 323 employed in the converter 212 shown in FIG. 22;

FIGS. 24A to I are timing charts used for explaining the operation ofthe converter 212 shown in FIG. 22;

FIG. 25 is a block diagram showing a typical configuration of aconverter 202 employed in the transcoder 101 shown in FIG. 15;

FIG. 26 is a block diagram showing a typical configuration of a deletecircuit 343 employed in the converter 202 shown in FIG. 25;

FIG. 27 is a block diagram showing another typical configuration of theconverter 212 employed in the transcoder 101 shown in FIG. 15;

FIG. 28 is a block diagram showing another typical configuration of theconverter 202 employed in the transcoder 101 shown in FIG. 15;

FIG. 29 is a block diagram showing a typical configuration of auser-data formatter 213 employed in the transcoder 101 shown in FIG. 15;

FIG. 30 is a block diagram showing the configuration of an actual systememploying a plurality of transcoders 101 each shown in FIG. 14;

FIG. 31 is a diagram showing areas for recording encoding parameters;

FIG. 32 is a flowchart used for explaining processing carried out by theencoding apparatus 106 employed in the transcoder 101 shown in FIG. 14to determine changeable picture types;

FIG. 33 is a diagram showing an example of changing picture types;

FIG. 34 is a diagram showing another example of changing picture types;

FIG. 35 is an explanatory diagram used for describing quantizationcontrol processing carried out by the encoding apparatus 106 employed inthe transcoder 101 shown in FIG. 14;

FIG. 36 is a flowchart used for explaining quantization controlprocessing carried out by the encoding apparatus 106 employed in thetranscoder 101 shown in FIG. 14;

FIG. 37 is a block diagram showing the configuration of a tightlycoupled transcoder 101;

FIG. 38 is an explanatory diagram used for describing the syntax of anMPEG stream;

FIG. 39 is an explanatory diagram used for describing the configurationof the syntax shown in FIG. 38;

FIG. 40 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a fixed length;

FIG. 41 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a fixed length;

FIG. 42 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a fixed length;

FIG. 43 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a fixed length;

FIG. 44 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a fixed length;

FIG. 45 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a fixed length;

FIG. 46 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a fixed length;

FIG. 47 is an explanatory diagram used for describing the syntax ofhistory_stream( ) for recording history information with a variablelength;

FIG. 48 is an explanatory diagram used for describing the syntax ofsequence_header( );

FIG. 49 is an explanatory diagram used for describing the syntax ofsequence_extension( );

FIG. 50 is an explanatory diagram used for describing the syntax ofextension_and_user_data( );

FIG. 51 is an explanatory diagram used for describing the syntax ofuser_data( );

FIG. 52 is an explanatory diagram used for describing the syntax ofgroup_of_picture_header( );

FIG. 53 is an explanatory diagram used for describing the syntax ofpicture_header( );

FIG. 54 is an explanatory diagram used for describing the syntax ofpicture_coding_extension( );

FIG. 55 is an explanatory diagram used for describing the syntax ofextension_data( );

FIG. 56 is an explanatory diagram used for describing the syntax ofquant_matrix_extension( );

FIG. 57 is an explanatory diagram used for describing the syntax ofcopyright_extension( );

FIG. 58 is an explanatory diagram used for describing the syntax ofpicture_display_extension( );

FIG. 59 is an explanatory diagram used for describing the syntax ofpicture_data( );

FIG. 60 is an explanatory diagram used for describing the syntax ofslice( );

FIG. 61 is an explanatory diagram used for describing the syntax ofmacroblock( );

FIG. 62 is an explanatory diagram used for describing the syntax ofmacroblock_modes( );

FIG. 63 is an explanatory diagram used for describing the syntax ofmotion_vectors(s);

FIG. 64 is an explanatory diagram used for describing the syntax ofmotion_vector(r, s);

FIG. 65 is an explanatory diagram used for describing a variable-lengthcode of macroblock_type for an I-picture;

FIG. 66 is an explanatory diagram used for describing a variable-lengthcode of macroblock_type for a P-picture; and

FIG. 67 is an explanatory diagram used for describing a variable-lengthcode of macroblock_type for a B-picture.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Before describing a transcoder provided by the present invention, aprocess to compress and encode a moving-picture video signal isexplained. It should be noted that a technical term ‘system’ used inthis specification means a whole system comprising a plurality ofapparatuses and means.

As described above, in a system for transmitting a moving-picture videosignal to a remote destination such as a television signal broadcastingsystem and a television signal transmitting system, the video-signal issubjected to compression and encoding processes using line correlationand intra-frame correlation of the video signal in order to allow thetransmission line to be utilized with a high degree of efficiency. Byusing line correlation, a video signal can be compressed by carrying outtypically DCT (Discrete Cosine Transform) processing.

By using intra-frame correlation, the video signal can be furthercompressed and encoded. Assume that frame pictures PC1, PC2 and PC3 aregenerated at points of time t1, t2 and t3 respectively as shown inFIG. 1. In this case, a difference in picture signal between the framepictures PC1 and PC2 is computed to generate a frame picture PC12. Bythe same token, a difference in picture signal between the framepictures PC2 and PC3 is computed to generate a frame picture PC23.Normally, a difference in picture signal between frame pictures adjacentto each other along the time axis is small. Thus, the amounts ofinformation contained in the frame pictures PC21 and PC23 are small andthe amount of code included in a difference signal obtained as a resultof coding such a difference is also small.

By merely transmitting a difference signal, however, the originalpicture can not be restored. In order to obtain the original picture,frame pictures are classified into three types, namely, I-, P- andB-pictures each used as a smallest processing unit in the compressionand encoding processes of a video signal.

Assume a GOP (Group of Pictures) of FIG. 2 comprising seventeen frames,namely, frames F1 to F17 which are each processed as a smallest unit ofvideo signals for processing. To be more specific, the first frame F1,the second frame F2 and the third frame F3 are processed as I-, B- andP-pictures respectively. The subsequent frames, that is, the fourth toseventeenth frames F4 to F17, are processed as B- and P-picturesalternately.

In the case of an I-picture, a video signal of the entire frame istransmitted. In the case of a P-picture or a B-picture, on the otherhand, only a difference in video signal can be transmitted as analternative to the video signal of the entire frame. To be morespecific, in the case of the third frame F3 of a P-picture shown in FIG.2, only a difference in video signal between the P-picture and achronically preceding I- or P-picture is transmitted as a video signal.In the case of the second frame F2 of a B-picture shown in FIG. 3, forexample, a difference in video signal between the B-picture and achronically preceding frame, a succeeding frame or an average value ofthe preceding and a chronically succeeding frames is transmitted as avideo signal.

FIG. 4 is a diagram showing the principle of a technique of encoding amoving-picture video signal in accordance with what is described above.As shown in FIG. 4, the first frame F1 is processed as an I-picture.Thus, the video signal of the entire frame F1 is transmitted to thetransmission line as data F1X (intra-picture encoding). On the otherhand, the second frame F2 is processed as a B-picture. In this case, adifference between the second frame F2 and the chronically precedingframe F1, the succeeding frame F3 or an average value of the precedingframe F1 and the succeeding frame F3 is transmitted as data F2X.

To put it in detail, the processing of a B-picture can be classifiedinto four types. In the processing of the first type, the data of theoriginal frame F2 denoted by notation SP1 in FIG. 4 is transmitted as itis as data F2X as is the same case with an I-picture. Thus, theprocessing of the first type is thus the so-called intra-pictureencoding. In the processing of the second type, a difference denoted bynotation SP2 between the second frame F2 and the chronically succeedingthird frame F3 is transmitted as data F2X. Since the succeeding frame istaken as a reference or a prediction picture, this processing isreferred to as backward prediction encoding. In the processing of thethird type, a difference denoted by notation SP3 between the secondframe F2 and the preceding first frame F1 is transmitted as data F2X asis the case with a P-picture. Since the preceding frame is taken as aprediction picture, this processing is referred to as forward predictionencoding. In the processing of the fourth type, a difference denoted bynotation SP4 between the second frame F2 and an average of thesucceeding third frame F3 and the preceding first frame F1 istransmitted as data F2X. Since the preceding and succeeding frames aretaken as a prediction picture, this processing is referred to as forward& backward prediction encoding. In actuality, one of the four processingtypes aforementioned is selected so as to generate a minimum amount oftransmission data obtained as a result of the processing.

It should be noted that, in the case of a difference obtained as aresult the processing of the second, third or fourth type describedabove, a motion vector between the pictures of the frames (predictionpicture) used in the computation of the difference is also transmittedalong the difference. To be mote specific, in the case of the forwardprediction encoding, the motion vector is a motion vector x1 between theframes F1 and F2. In the case of the backward prediction encoding, themotion vector is a motion vector x2 between the frames F2 and F3. In thecase of the forward & backward prediction encoding, the motion vectorsx1 and x2 are both transmitted.

Much like the B-picture described above, in the case of the frame F3 ofthe P-picture, the forward prediction encoding or the intra-pictureprocessing is selected to produce a minimum amount of transmitting dataobtained as a result of the processing. If the forward predictionencoding is selected, a difference denoted by notation SP3 between thethird frame F3 and the preceding first frame F1 is transmitted as dataF3X along with a motion vector x3. If the intra-picture processing isselected, on the other hand, it is the data F3X of the original frame F3denoted by notation SP1 that is transmitted.

FIG. 5 is a block diagram showing a typical configuration of a systembased on the principle described above to code a moving-picture videosignal and transmit as well as decode the coded signal. A signalencoding apparatus 1 encodes an input video signal and transmits theencoded video signal to a signal decoding apparatus 2 through arecording medium 3 which serves as a transmission line. The signaldecoding apparatus 2 plays back the coded signal recorded on therecording medium 3 and decodes the playback signal in to an outputsignal.

In the signal encoding apparatus 1, the input video signal is suppliedto a preprocessing circuit 11 for splitting it into luminance andchrominance signals. In the case of this embodiment, the chrominancesignal is a color-difference signal. The analog luminance andcolor-difference signals are then supplied to A/D converters 12 and 13respectively to be each converted into a digital video signal. Digitalvideo signals resulting from the A/D conversion are then supplied to aframe-memory unit 14 to be stored therein. The frame-memory unit 14comprises a luminance-signal frame memory 15 for storing the luminancesignal and a color-difference signal frame memory 16 for storing thecolor-difference signal.

A format converting circuit 17 converts the frame-format signals storedin the frame-memory unit 14 into a block-format signal as shown in FIGS.6A to 6C. To put it in detail, a video signal is stored in theframe-memory unit 14 as data of the frame format shown in FIG. 6A. Asshown in FIG. 6A, the frame format is a collection of V lines eachcomprising H dots. The format converting circuit 17 divides the signalof one frame into N slices each comprising 16 lines as shown in FIG. 6B.Each slice is then divided into M pieces of macroblocks as shown in FIG.6B. As shown in FIG. 6C, a macroblock includes a luminance signal Ycorresponding to 16×16 pixels (dots). The luminance signal Y is furtherdivided into blocks Y[1] to Y[4] each comprising 8×8 dots. The 16×16-dotluminance signal is associated with a 8×8-dot Cb signal and, a 8×8-dotCr signal.

The data with the block format obtained as a result of the formatconversion carried out by the format converting circuit 17 as describedabove is supplied to an encoder 18 for encoding the data. Theconfiguration of the encoder 18 will be described later in detail byreferring to FIG. 7.

A signal obtained as a result of the encoding carried out by the encoder18 is output to a transmission line as a bitstream. Typically, theencoded signal is supplied to a recording circuit 19 for recording theencoded signal onto a recording medium 3 used as a transmission line asa digital signal.

A playback circuit 30 employed in the signal decoding apparatus 2reproduces data from the recording medium 3, supplying the data to adecoder 31 of the decoding apparatus for decoding the data. Theconfiguration of the decoder 31 will be described later in detail byreferring to FIG. 12.

Data obtained as a result of the decoding carried out by the decoder 31is supplied to a format converting circuit 32 for converting the blockformat of the data back into a frame format. Then, a luminance signalhaving a frame format is supplied to a luminance-signal frame memory 34of a frame-memory unit 33 to be stored therein. On the other hand, acolor-difference signal having a frame format is supplied to acolor-difference-signal frame memory 35 of the frame-memory unit 33 tobe stored therein. The luminance signal is read back from theluminance-signal frame memory 34 and supplied to a D/A converter 36. Onthe other hand, the color-difference signal is read back from thecolor-difference-signal frame memory 35 and supplied to a D/A converter37. The D/A converters 36 and 37 convert the signals into analog signalswhich are then supplied to a post-processing circuit 38 for synthesizingthe luminance and color-difference signals and generating a synthesizedoutput.

Next, the configuration of the encoder 18 is described by referring toFIG. 7. Picture data to be encoded is supplied to a motion-vectordetecting circuit 50 in macroblock units. The motion-vector detectingcircuit 50 processes picture data of each frame as an I-, P- orB-picture in accordance with a predetermined sequence set in advance. Tobe more specific, picture data of a GOP typically comprising frames theF1 to F17 as shown in FIGS. 2 and 3 is processed as a sequence of I-,B-, P-, B-, P-, - - - , B- and P-pictures.

Picture data of a frame to be processed by the motion-vector detectingcircuit 50 as an I-picture such as the frame F1 shown in FIG. 3 issupplied to a forward-source-picture area 51 a of a frame-memory unit 51to be stored therein. Picture data of a frame to be processed by themotion-vector detecting circuit 50 as a P-picture such as the frame F2is supplied to a referenced-source-picture area 51 b of the frame-memoryunit 51 to be stored therein. Picture data of a frame processed by themotion-vector detecting circuit 50 as a P-picture such as the frame F3is supplied to a backward-source-picture area 51 c of the frame-memoryunit 51 to be stored therein.

When picture data of the next two frames such as the frame F4 or F5 aresupplied sequentially to the motion-vector detecting circuit 50 to beprocessed as B and P-pictures respectively, the areas 51 a, 51 b and 51c are updated as follows. When picture data of the frame F4 is processedby the motion-vector detecting circuit 50, the picture data of the frameF3 stored in the backward-source-picture area 51 c is transferred to theforward-source-picture area 51 a, overwriting the picture data of theframe F1 stored earlier in the source-picture area 51 b. The processedpicture data of the frame F4 is stored in the referenced-source-picturearea 51 b, overwriting the picture data on the frame F2 stored earlierin the referenced-source-picture area 51 b. Then, the processed picturedata of the frame F5 is stored in the backward-source-picture area 51 c,overwriting the picture data of the frame F3 which has been transferredto the forward-source-picture area 51 a any way. The operationsdescribed above are repeated to process the subsequent frames of theGOP.

Signals of each picture stored in the frame-memory unit 51 are read outby a prediction-mode switching circuit 52 to undergo a preparatoryoperation for a frame-prediction mode or a field-prediction mode, thatis, the type of processing to be carried out by a processing unit 53.

Then, in either the frame-prediction mode or the field-prediction mode,the signals are subjected to computing to obtain intra-pictureprediction encoding, such as, forward, backward and forward & backwardprediction under control executed by anintra-picture-processing/forward/backward/forward & backward predictiondetermining circuit 54. The type of processing carried out in theprocessing unit 53 is determined in accordance with a prediction-errorsignal representing a difference between a referenced picture and aprediction picture for the referenced picture. A referenced picture is apicture subjected to the processing and a prediction picture is apicture preceding or succeeding the referenced picture. For this reason,the motion-vector detecting circuit 50 (strictly speaking, theprediction-mode switching circuit 52 employed in the vector detectingcircuit 50 as will be described later) generates a sum of the absolutevalues of prediction-error signals for use in determination of the typeof processing carried out by the processing unit 53. In place of a sumof the absolute values of prediction-error signals, a sum of the squaresof prediction-error signals can also be used for such determination.

The prediction-mode switching circuit 52 carries out the followingpreparatory operation for processing to be carried out by the processingunit 53 in the frame prediction mode and the field prediction mode.

The prediction-mode switching circuit 52 receives four luminance blocks[Y1] to [Y4] supplied thereto, by the motion-vector detecting circuit50. In each block, data of lines of odd fields is mixed with data oflines of even fields as shown in FIG. 8. The data may be passed on tothe processing unit 53 as it is. Processing of data with a configurationshown in FIG. 8 to be performed by the processing unit 53 is referred toas processing in the frame-prediction mode wherein prediction processingis carried out for each macroblock comprising the four luminance blocksand a motion vector corresponds to four luminance blocks.

The prediction-mode switching circuit 52 then reconfigures the signalsupplied by the motion-vector detecting circuit 50. In place of thesignal with the configuration shown in FIG. 8, the signal with theconfiguration shown in FIG. 9 may be passed on to the processing unit53. As shown in FIG. 9, the two luminance blocks [Y1] and [Y2] are eachcomposed of typically only dots of lines of odd fields whereas the othertwo luminance blocks [Y3] and [Y4] are each composed of typically onlydots of lines of even fields. Processing of data with the configurationshown in FIG. 9 to be carried out by the processing unit 53 is referredto as processing in the field-prediction mode wherein a motion vectorcorresponds to the two luminance blocks [Y1] and [Y2] whereas anothermotion vector corresponds to the other two luminance blocks [Y3] and[Y4].

The prediction-mode switching circuit 52 selects the data with theconfiguration shown in FIG. 8 or FIG. 9 to be supplied to the processingunit 53 as follows. The prediction-mode switching circuit 52 computes asum of the absolute values of prediction errors computed for theframe-prediction mode, that is, for the data supplied by themotion-vector detecting circuit 50 with the configuration shown in FIG.8, and a sum of the absolute values of prediction errors computed forthe field-prediction mode, that is, for the data with the configurationshown in FIG. 9 obtained as a result of conversion of the data with theconfiguration shown in FIG. 8. It should be noted that the predictionerrors will be described in detail later. The prediction-mode switchingcircuit 52 then compares the sum of the absolute values of predictionerrors computed for data in order to determine which mode produces thesmaller sum. Then, the prediction-mode switching circuit 52 selectseither one of the configuration shown in FIG. 8 or FIG. 9 for theframe-prediction mode or the field-prediction mode respectively thatproduces the smaller sum. The prediction-mode switching circuit 52finally outputs data with the selected configuration to the processingunit 53 for processing the data in the mode corresponding to theselected configuration.

It should be noted that, in actuality, the prediction-mode switchingcircuit 52 is included in the motion-vector detecting circuit 50. Thatis to say, the preparation of data with the configuration shown in FIG.9, the computation of the absolute values, the comparison of theabsolute values, the selection of the data configuration and theoperation to output data with the selected configuration to theprocessing unit 53 are all carried out by the motion-vector detectingcircuit 50 and the prediction-mode switching circuit 52 merely outputsthe signal supplied by the motion-vector detecting circuit 50 to thelater stage of processing unit 53.

It should be noted that, in the frame-prediction mode, acolor-difference signal is supplied to the processing unit 53 with dataof lines of odd fields mixed with data of lines of even fields as shownin FIG. 8. In the field-prediction mode shown in FIG. 9, on the otherhand, four lines of the upper half of the color-difference block Cb areused as a color-difference signal of odd fields corresponding to theluminance blocks [Y1] and [Y2] while four lines of the lower half of thecolor-difference block Cb are used as a color-difference signal of evenfields corresponding to the luminance blocks [Y3] and [Y4]. By the sametoken, four lines of the upper half of the color-difference block Cr areused as a color-difference signal of odd fields corresponding to theluminance blocks [Y1] and [Y2] while four lines of the lower half of thecolor-difference block Cr are used as a color-difference signal of evenfields corresponding to the luminance blocks [Y3] and [Y4].

As described above, the motion-vector detecting circuit 50 outputs a sumof the absolute values of prediction errors to the predictiondetermining circuit 54 for use in determination of whether theprocessing unit 53 should carry out intra-picture-prediction, forwardprediction, backward prediction or forward & backward prediction.

To put it in detail, a sum of the absolute values of prediction errorsin intra-picture prediction is found as follows. For the intra-pictureprediction, the motion-vector detecting circuit 50 computes a differencebetween the absolute value |Ai j| of a sum Aij of signals Aij ofmacroblocks of a referenced picture and a sum |Aij| of the absolutevalues |Aij| of the signals Aij of the macroblocks of the samereferenced picture. For the forward prediction, the sum of the absolutevalues of prediction errors is a sum |A ij−Bij| of the absolute values|Aij−Bij| of differences (Aij−Bij) between signals Aij of macroblocks ofa referenced picture and signals Bij of macroblocks of aforward-prediction picture or a preceding picture. The sum of theabsolute values of prediction errors for the backward prediction isfound in the same way as that for the forward prediction, except thatthe prediction picture used in the backward prediction is abackward-prediction picture or a succeeding picture. As for the forward& backward prediction, averages of signals Bij of macroblocks of boththe forward-prediction picture and the backward-prediction picture areused in the computation of the sum.

The sum of the absolute values of prediction errors for each techniqueof prediction is supplied to the prediction determining circuit 54 whichselects the forward prediction, the backward prediction or the forward &backward prediction with a smallest sum as a sum of absolute value ofprediction error for an inter-picture prediction. The predictiondetermining circuit 54 further compares the smallest sum with the sumfor the intra-picture prediction and selects either the inter-pictureprediction or the intra-picture prediction with the much smaller sum asa prediction mode of processing to be carried out by the processing unit53. To be more specific, if the sum for the intra-picture prediction isfound smaller than the smallest sum for the inter-picture prediction,the prediction determining circuit 54 selects the intra-pictureprediction as a type of processing to be carried out by the processingunit 53. If the smallest sum for the inter-picture prediction is foundsmaller than the sum for the intra-picture prediction, on the otherhand, the prediction determining circuit 54 selects the inter-pictureprediction as a type of processing to be carried out by the processingunit 53. As described above, the inter-picture prediction represents theforward prediction, the backward prediction or the forward & backwardprediction selected as a prediction mode of processing with the smallestsum. The prediction mode is determined for each picture (or frame) whilethe frame-prediction or field-prediction mode is determined for eachgroup of pictures.

As described above, the motion-vector detecting circuit 50 outputssignals of macroblocks of a referenced picture for either theframe-prediction mode or the field-prediction mode selected by theprediction-mode switching circuit 52 to the processing unit 53 by way ofthe prediction-mode switching circuit 52. At the same time, themotion-vector detecting circuit 50 also detects a motion vector betweena referenced picture and a prediction picture for one of the fourprediction modes selected by the prediction determining circuit 54. Themotion-vector detecting circuit 50 then outputs the motion vector to avariable-length-encoding circuit 58 and a motion-compensation circuit64. In this way, the motion-vector detecting circuit 50 outputs a motionvector that corresponds to the minimum sum of the absolute values ofprediction errors for the selected prediction mode as described earlier.

While the motion-vector detecting circuit 50 is reading out picture dataof an I-picture, the first frame of a GOP, from the forward sourcepicture area 51 a, the prediction determining circuit 54 sets theintra-picture prediction, strictly speaking, the intra-frame orintra-field prediction, as a prediction mode, setting a switch 53 demployed in the processing unit 53 at a contact point a. With the switch53 d set at this position, the data of the I-picture is supplied to aDCT-mode switching circuit 55. As will be described later, theintra-picture prediction mode is a mode in which no motion compensationis carried out.

The DCT-mode switching circuit 55 receives the data passed on thereto bythe switch 53 d from the prediction-mode switching circuit 52 in a mixedstate or a frame-DCT mode shown FIG. 10. The DCT-mode switching circuit55 then converts the data into a separated state or a field-DCT modeshown in FIG. 11. In the frame-DCT mode, the data of lines of odd andeven fields are mixed in each of the four luminance blocks. In thefield-DCT mode, on the other hand, lines of odd fields are put in two ofthe four luminance blocks while lines of even fields are put in theother two blocks. The data of the I-picture in either the mixed orseparated state will be supplied to a DCT circuit 56.

Before supplying the data to the DCT circuit 56, the DCT-mode switchingcircuit 55 compares the encoding efficiency of DCT processing of thedata with the lines of odd and even fields mixed with each other withthe encoding efficiency of DCT processing of the data with the lines ofodd and even fields separated from each other to select the data with ahigher efficiency. The frame-DCT mode or the field-DCT modecorresponding to the selected data is determined as the DCT mode.

The encoding efficiencies are compared with each other as follows. Inthe case of the data with the lines of odd and even fields mixed witheach other as shown in FIG. 10, a difference between the signal of aline of an even field and the signal of a line of an odd fieldvertically adjacent to the even field is computed. Then, the sum or thesquare of the absolute value of the difference is found. Finally, thesum of the absolute values or the squares of differences between twoadjacent even and odd fields is calculated.

In the case of the data with the lines of odd and even fields separatedfrom each other as shown in FIG. 11, a difference between the signals oflines of vertically adjacent even fields and a difference between thesignals of lines of vertically adjacent odd fields are computed. Then,the sum or the square of the absolute value of each difference is found.Finally, the sum of the absolute values or the squares of alldifferences between two adjacent even fields and two adjacent odd fieldsis calculated.

The sum calculated for the data shown in FIG. 10 is compared with thesum calculated for the data shown in FIG. 11 to select a DCT mode. To bemore specific, if the former is found smaller than the latter, theframe-DCT mode is selected. If the latter is found smaller than theformer, on the other hand, the field-DCT mode is selected.

Finally, the data with a configuration corresponding to the selected DCTmode is supplied to the DCT circuit 56, and at the same time, a DCT flagused for indicating the selected DCT mode is supplied to thevariable-length-encoding circuit 58 and the motion-compensation circuit64.

Comparison of the frame-prediction and field-prediction modes of FIGS. 8and 9 determined by the prediction-mode switching circuit 52 with theDCT modes of FIGS. 10 and 11 determined by the DCT-mode switchingcircuit 55 clearly indicates that the data structures of theframe-prediction and field-prediction modes are substantially the sameas the data structures of the frame-DCT mode and field-DCT modesrespectively with regards to the luminous block.

If the prediction-mode switching circuit 52 selects the frame-predictionmode in which odd and even lines are mixed with each other, it is quitewithin the bounds of possibility that the DCT-mode switching circuit 55also selects the frame-DCT prediction mode with odd and even linesmixed. If the prediction-mode switching circuit 52 selects thefield-prediction mode in which odd and even fields are separated fromeach other, it is quite within the bounds of possibility that theDCT-mode switching circuit 55 also selects the field-DCT mode with thedata of odd and even fields separated.

It should be noted, however, that a selected DCT-mode does not alwayscorrespond to a selected prediction mode. In any circumstances, theprediction-mode switching circuit 52 selects a frame-prediction orfield-prediction mode that provides a smaller sum of the absolute valuesof prediction errors and the DCT-mode switching circuit 55 selects a DCTmode giving a better encoding efficiency.

As described above, the data of the I-picture is output by the DCT-modeswitching circuit 55 to the DCT circuit 56. The DCT circuit 56 thencarries out DCT processing on the data for conversion into DCTcoefficients which are then supplied to a quantization circuit 57. Thequantization circuit 57 then carries out a quantization process at aquantization scale adjusted to the amount of data stored in atransmission buffer 59 which is fed back to the quantization circuit 57as will be described later. The data of the I-picture completing thequantization process is then supplied to a variable-length-encodingcircuit 58.

The variable-length-encoding circuit 58 receives the data of theI-picture supplied from the quantization circuit 57, converting thepicture data into variable-length code such as the Huffman code, at thequantization scale supplied thereto also by the quantization circuit 57.The variable-length code is then stored in the transmission buffer 59.

In addition to the picture data and the quantization scale supplied bythe quantization circuit 57, the variable-length-encoding circuit 58also receives information on a prediction mode from the predictiondetermining circuit 54, a motion vector from the motion-vector detectingcircuit 50, a prediction flag from the prediction-mode switching circuit52 and a DCT flag from the DCT-mode switching circuit 55. Theinformation on a prediction mode indicates which type of processing iscarried out by the processing unit among intra-picture encoding,forward-prediction encoding, backward-prediction encoding or forward &backward-prediction encoding. The prediction flag indicates that datasupplied from the prediction-mode switching circuit 52 to the processingunit 53 is whether in the frame-prediction mode or field-predictionmode. The DCT flag indicates that data supplied from the DCT-modeswitching circuit 55 to the DCT circuit 56 is set whether in theframe-DCT or field-DCT mode.

The transmission buffer 59 temporarily stores input data, feeding theamount of the data stored therein back to the quantization circuit 57.When the amount of data stored in the transmission buffer 59 exceeds anupper limit of an allowable range, the quantization control signalincreases the quantization scale of the quantization circuit 57 toreduce the amount of data obtained as a result of the quantization. Whenthe amount of data stored in the transmission buffer 59 becomes smallerthan a lower limit of the allowable range, on the other hand, thequantization control signal decreases the quantization scale of thequantization circuit 57 to raise the amount of data obtained as a resultof the quantization. In this way, an overflow and an underflow can beprevented from occurring in the transmission buffer 59.

Then, the data stored in the transmission buffer 59 is read back atpredetermined timing to be supplied to the recording circuit 19 forrecording the data onto the recording medium 3 which serves as atransmission line.

The data of the I-picture and the quantization scale output by thequantization circuit 57 are supplied also to an inverse-quantizationcircuit 60 for carrying out inverse quantization on the data at aninverse-quantization scale corresponding to the quantization scale. Dataoutput by the inverse-quantization circuit 60 is then supplied to anIDCT (Inverse Discrete Cosine Transform) circuit 61 for carrying outinverse discrete cosine transformation. Finally, data output by the IDCTcircuit 61 is supplied to a frame-memory unit 63 by way of a processor62 to be stored in a forward-prediction picture area 63 a of theframe-memory unit 63.

A GOP supplied to the motion-picture detecting circuit 50 to beprocessed thereby comprises a sequence of pictures I-, B-, P-, B-, P-,B-, etc. In this case, after processing data of the first frame as anI-picture as described above, data of the third frame is processed as aP-picture prior to processing of data of the second frame as aB-picture. This is because a B-picture may be subjected to backwardprediction which involves the succeeding P-picture and backwardprediction can not be carried out unless the succeeding P-picture hasbeen processed in advance. It should be noted that the data of theI-picture is transferred from the prediction-mode switching circuit 52to the processing unit 53 in the format for the frame-prediction orfield-prediction mode set by the prediction-mode switching circuit 52for the GOP and always processed by the processing unit 53 in theintra-frame prediction mode as described earlier.

For the reason described above, after processing data of the first frameas an I-picture, the motion-picture detecting circuit 50starts-processing of the P-picture which has been stored in thebackward-source-picture area 51 c. Then, the prediction-mode switchingcircuit 52 computes a sum of the absolute values of differences betweenframes or prediction errors for data of the P-picture supplied theretoby the motion-vector detecting circuit 50 with the macroblock taken as aunit for each prediction mode and supplies the sum to the predictiondetermining circuit 54 as described above. The data of the P-pictureitself is transferred to the processing unit 53 in the format for theframe-prediction or field-prediction mode which has been set by theprediction-mode switching circuit 52 for the GOP of the P-picture whenthe I-picture of the first frame of the GOP was input. On the otherhand, the prediction determining circuit 54 determines the predictionmode in which the data of the P-picture is to be processed by theprocessing unit 53, that is, selects either the intra-picture, forward,backward or forward & backward prediction as the type of processing tobe carried out by the processing unit 53 on the data of the P-picture onthe basis of the sum of the absolute values of prediction errorscomputed by the prediction-error switching circuit 52 for eachprediction mode. Strictly speaking, in the case of a P-picture, the typeof processing can be the intra-picture or forward-prediction mode asdescribed above.

In the first place, in the intra-picture prediction mode, the processingunit 53 sets the switch 53 d at the contact point a. Thus, the data ofthe P-picture is transferred to the transmission line by way of theDCT-mode switching circuit 55, the DCT circuit 56, the quantizationcircuit 57, the variable-length-encoding circuit 58 and the transmissionbuffer 59 as is the case with an I-picture. The data of the P-picture isalso supplied to the frame-memory unit 63 to be stored in thebackward-prediction picture area 63 b thereof by way of the quantizationcircuit 57, the inverse-quantization circuit 60, the IDCT circuit 61 andthe processor 62.

In the second place, in the forward-prediction mode, the processing unit53 sets the switch 53 d at a contact point b and the motion-compensationcircuit 64 reads out data from the forward-prediction picture area 63 aof the frame-memory unit 63, carrying out motion compensation on thedata in accordance with a motion vector supplied to themotion-compensation circuit 64 by the motion-vector detecting circuit50. In this case, the data stored in the forward-prediction picture area63 a is the data of the I-picture. That is to say, informed of theforward-prediction mode by the prediction determining circuit 54, themotion-compensation circuit 64 generates data of a forward-predictionpicture data by reading the data of the I-picture from a read address inthe forward-prediction picture area 63 a. The read address is a positionshifted away from the position of a macroblock currently output by themotion-vector detecting circuit 50 by a distance corresponding to themotion vector.

The data of the forward-prediction picture read out by themotion-compensation circuit 64 is associated with the data of thereferenced picture, that is, the P-picture, and supplied to a processor53 a employed in the processing unit 53. The processor 53 a subtractsthe data of the forward-prediction picture, that is, the I-picture,supplied by the motion-compensation circuit 64 from the data ofmacroblocks of the referenced picture supplied by the prediction-modeswitching circuit 52 to find a difference or an error in prediction. Thedifference data is transferred to the transmission line by way of theDCT-mode switching circuit 55, the DCT circuit 56, the quantizationcircuit 57, the variable-length-encoding circuit 58 and the transmissionbuffer 59. The difference data is also locally decoded by theinverse-quantization circuit 60 and the IDCT circuit 61 and a resultobtained from the local decoding is supplied to the processor 62.

The data of the forward-prediction picture supplied to the processor 53a by the motion-compensation circuit 64 is also fed to the processor 62.In the processor 62, the data of the forward-prediction picture is addedto the difference data output by the IDCT circuit 61 in order to producethe data of the original P-picture. The data of the original P-pictureis then stored in the backward-prediction picture area 63 b of theframe-memory unit 63.

After the pieces of data of the I-picture and the P-picture have beenstored in the forward-prediction picture area 63 a and thebackward-prediction picture area 63 b of the frame-memory unit 63respectively as described above, the processing of the second frame ofthe B-picture is started by the motion-vector detecting circuit 50. TheB-picture is processed by the prediction-mode switching circuit 52 inthe same way as the P-picture described above except that, in the caseof the B-picture, the type of processing determined by the predictiondetermining circuit 54 can be the backward-prediction mode or theforward & backward-prediction mode in addition to the intra-pictureprediction mode or the forward-prediction mode.

In the case of the intra-picture prediction-mode or theforward-prediction mode, the switch 53 d is set at the contact point aor b respectively as described above as is the case with the P-picture.In this case, data of the B-picture output by the prediction-modeswitching circuit 52 is processed and transferred in the same way as theP-picture described above.

In the case of the backward-prediction mode or the forward &backward-prediction mode, on the other hand, the switch 53 d is set at acontact point c or d respectively.

In the backward-prediction mode wherein the switch 53 d is set at thecontact point c, the motion-compensation circuit 64 reads out data fromthe backward-prediction picture area 63 b of the frame-memory unit 63,carrying out motion compensation on the data in accordance with a motionvector supplied to the motion-compensation circuit 64 by themotion-vector detecting circuit 50. In this case, the data stored in thebackward-prediction picture area 63 b is the data of the P-picture. Thatis to say, informed of the backward-prediction mode by the predictiondetermining circuit 54, the motion-compensation circuit 64 generatesdata of a backward-prediction picture by reading the data of theP-picture from a read address in the backward-prediction picture area 63b. The read address is a position shifted away from the position of amacroblock currently output by the motion-vector detecting circuit 50 bya distance corresponding to the motion vector.

The data of the backward-prediction picture read out by themotion-compensation circuit 64 is associated with the data of areferenced picture, that is, the B-picture, and supplied to a processor53 b employed in the processing unit 53. The processor 53 b subtractsthe data of the backward-prediction picture, that is, the P-picture,supplied by the motion-compensation circuit 64 from the data ofmacroblocks of the referenced picture supplied by the prediction-modeswitching circuit 52 to find a difference or an error in prediction. Thedifference data is transferred to the transmission line by way of theDCT-mode switching circuit 55, the DCT circuit 56, the quantizationcircuit 57, the variable-length-encoding circuit 58 and the transmissionbuffer 59.

In the forward & backward-prediction mode wherein the switch 53 d is setat the contact point d, on the other hand, the motion-compensationcircuit 64 reads out the data of the I-picture, in this case, from theforward-prediction picture area 63 a of the frame-memory unit 63 and thedata of the P-picture from the backward-prediction picture area 63 b,carrying out motion compensation on the data in accordance with motionvectors supplied to the motion-compensation circuit 64 by themotion-vector detecting circuit 50.

That is to say, informed of the forward & backward-prediction mode bythe prediction determining circuit 54, the motion-compensation circuit64 generates data of forward & backward-prediction pictures by readingthe data of the I- and P-pictures from read addresses in theforward-prediction picture area 63 a and the backward-prediction picturearea 63 b respectively. The read addresses are positions shifted awayfrom the position of a macroblock currently output by the motion-vectordetecting circuit 50 by distances corresponding to the motion vectors.In this case, there are 2 motion vectors, namely, motion vectors for theforward & backward-prediction pictures.

The data of the forward & backward-prediction pictures read out by themotion-compensation circuit 64 is associated with the data of thereferenced picture and supplied to a processor 53 c employed in theprocessing unit 53. The processor 53 c subtracts the data of theprediction pictures supplied by the motion-compensation circuit 64 fromthe data of macroblocks of the referenced picture supplied by theprediction-mode switching circuit 52 employed in the motion-vectordetecting circuit 50 to find a difference or an error in prediction. Thedifference data is transferred to the transmission line by way of theDCT-mode switching circuit 55, the DCT circuit 56, the quantizationcircuit 57, the variable-length-encoding circuit 58 and the transmissionbuffer 59.

Never used as a prediction picture for another frame, a B-picture is notstored in the frame-memory unit 63.

It should be noted that, typically, the forward-prediction picture area63 a and the backward-prediction picture area 63 b of the frame-memoryunit 63 are implemented as memory banks which can be switched from oneto another. Thus, in an operation to read a forward-prediction picture,the frame-memory unit 63 is set to the forward-prediction picture area63 a. In an operation to read a backward-prediction picture, on theother hand, the frame-memory unit 63 is set to the backward-predictionpicture area 63 b.

While the above description is focused on luminance blocks, thecolor-difference blocks are also processed and transferred in macroblockunits shown in FIGS. 8 to 11 in the same way as the luminance blocks. Itshould be noted that, as a motion vector in the processing of thecolor-difference blocks, components of the motion vector of theluminance blocks associated with the color-difference blocks in thevertical and horizontal directions are used with their magnitudes eachcut in half.

FIG. 12 is a block diagram showing the configuration of the decoder 31employed in the moving-picture encoding/decoding apparatus shown in FIG.5. Encoded picture data transmitted through the transmission lineimplemented by the recording medium 3 is received by the decoder 31, byway of the playback circuit 30 of the moving-picture encoding/decodingapparatus and then stored temporarily in a reception buffer 81 employedin the decoder 31. Then, the picture data is supplied to avariable-length-decoding circuit 82 employed in a decoding circuit 90 ofthe decoder 31. The variable-length-decoding circuit 82 carries outvariable-length decoding on the picture data read out from the receptionbuffer 81, outputting a motion vector, information on the predictionmode, a frame/field-prediction flag and a frame/field DCT-flag to amotion compensation-circuit 87 and a quantization scale as well asdecoded picture data to an inverse-quantization circuit 83.

The inverse-quantization circuit 83 carries out inverse quantization onthe picture data supplied thereto by the variable-length-decodingcircuit 82 at the quantization scale also received from thevariable-length-decoding circuit 82. The inverse-quantization circuit 83outputs DCT coefficients data obtained as a result of the inversequantization to an IDCT circuit 84 for carrying out IDCT (inversediscrete cosine transformation), supplying the result of the IDCT to aprocessor 85.

In the case of an I-picture, the picture data supplied to the processor85 by the IDCT circuit 84 is output as it is, by the processor 85 to aframe-memory unit 86 to be stored in a forward-prediction picture area86 a of the frame-memory unit 86. The data of the I-picture stored inthe forward-prediction picture area 86 a will be used for generation ofdata of a forward-prediction picture for picture data of a P- orB-picture supplied to the processor 85 after the I-picture in theforward-prediction mode. The data of the I-picture is also output to aformat converting circuit 32 employed in the moving-pictureencoding/decoding apparatus shown in FIG. 5.

When the picture data supplied by the IDCT circuit 84 is the data ofP-picture having the picture data leading ahead by one frame, that is,the picture data of the I-picture, is read out by themotion-compensation circuit 87 from the forward-prediction picture area86 a of the frame-memory unit 86. In the motion-compensation circuit 87,the picture data of the I-picture undergoes motion compensation for amotion vector supplied by the variable-length-decoding circuit 82. Thepicture data completing the motion compensation is then supplied to theprocessor 85 to be added to the picture data supplied by the IDCTcircuit 84 which is actually difference data. The result of theaddition, that is, data of the decoded P-picture is supplied to theframe-memory unit 86 to be stored in the backward-prediction picturearea 86 b of the frame-memory unit 86 as described above. The data ofthe P-picture stored in the backward-prediction picture area 86 b willbe used for generation of data of a backward-prediction picture forpicture data of a B-picture supplied to the processor 85 thereafter inthe backward-prediction mode.

On the other hand, picture data of a P-picture processed by the signalencoding apparatus 1 in the intra-frame prediction mode is output by theprocessor 85 as it is without undergoing any processing, to thebackward-prediction picture area 86 b as is the case with an I-picture.

Since the P-picture will be displayed after a B-picture to be processedafter the P-picture, at this point of time, the P-picture is not outputto the format converting circuit 32. Much like the encoder 18, thedecoder 31 processes and transfers the P-picture prior to the B-pictureeven though the P-picture is received after the B-picture.

Picture data of a B-picture output by the IDCT circuit 84 is processedby the processor 85 in accordance with the information on the predictionmode supplied by the variable-length-decoding circuit 82. To be morespecific, the processor 85 may output the picture data as it is in theintra-picture-prediction mode as is the case with the I-picture orprocess the picture data in the forward-prediction, backward-predictionor forward & backward-prediction mode. In the forward-predictionbackward-prediction or forward & backward-prediction mode, themotion-compensation circuit 87 reads out the data of the I-picturestored in the forward-prediction picture area 86 a, the data of theP-picture stored in the backward-prediction picture area 86 b or thedata of the I and P-pictures stored in the forward-prediction picturearea 86 a and the backward-prediction picture area 86 b of theframe-memory unit 86 respectively. The motion-compensation circuit 87then carries out motion compensation on the picture data read out fromthe frame-memory unit 86 in accordance with a motion vector output bythe variable-length-decoding circuit 82 to produce a prediction picture.In the case of the intra-picture-prediction mode described above, noprediction picture is generated because no prediction picture isrequired by the processor 85.

The prediction picture experiencing the motion compensation in themotion-compensation circuit 87 is added by the processor 85 to thepicture data of the B-picture, strictly speaking difference data, outputby the IDCT circuit 84. Data output by the processor 85 is then suppliedto the format converting circuit 32 as is the case with the I-picture.

Since the data output by the processor 85 is the picture data of aB-picture, however, the data is not required in generation of aprediction picture. Thus, the data output by the processor 85 is notstored in the frame-memory unit 86.

After the data of the B-picture has been output, the data of theP-picture is read out by the motion-compensation circuit 87 from thebackward-prediction picture area 86 b and supplied to the processor 85.This time, however, the data is not subjected to motion compensationsince the data has experienced the motion compensation before beingstored in the backward-prediction picture area 86 b.

The decoder 31 does not include counterpart circuits of theprediction-mode switching circuit 52 and the DCT-mode switching circuit55 employed in the encoder 18 shown in FIG. 5 because the counterpartprocessing, that is, processing to convert the signal formats with evenand odd fields separated from each other as shown in FIGS. 9 and 11 backinto the signal formats with even and odd fields mixed with each otheras shown in FIGS. 8 and 10 respectively is carried out by themotion-compensation circuit 87.

While the above description is focused on luminance signals, thecolor-difference signals are also processed and transferred inmacroblock units shown in FIGS. 8 to 11 in the same way as the luminancesignals. It should be noted that, as a motion vector in the processingof the color-difference signals, components of the motion vector of theluminance signals associated with the color-difference blocks in thevertical and horizontal directions are used with their magnitudes eachcut in half.

FIG. 13 is a diagram showing the quality of encoded pictures in terms ofan SNR (Signal-to-Noise Ratio). As shown in the figure, the quality of apicture much depends on the type of the picture. To be more specific,transmitted I- and P-pictures have high qualities, but B-picture haspoorer qualities respectively. Deliberate variations in picture qualityshown in FIG. 13 is a sort of technique for making use of the humanbeing's characteristic of the sense of sight. That is to say, by varyingthe quality, the quality appears better to the sense of sight than acase in which an average of all picture qualities is used. Control tovary the picture quality is executed by the quantization circuit 57employed in the encoder 18 shown in FIG. 7.

FIGS. 14 and 15 are diagrams showing the configuration of the transcoder101 provided by the present invention. FIG. 15 shows the configurationshown in FIG. 14 in more detail. The transcoder 101 converts the GOPstructure and the bit rate of an encoded video bitstream supplied to thevideo decoding apparatus 102 into a GOP structure and a bit rate desiredby the operator or specified by the host computer, respectively. Thefunction of the transcoder 101 is explained by assuming that, inactuality, three other transcoders each having all but the same functionas the transcoder 101 are connected at the front stage of the transcoder101. In order to convert the GOP structure and the bit rate of bitstreaminto one of a variety of GOP structures and one of a variety of bitrates respectively, transcoders of the first, second and thirdgenerations are connected in series and the transcoder 101 of the fourthgeneration shown in FIG. 15 is connected behind the series connection ofthe transcoders of the first, second and third generation. It should benoted that the transcoders of the first, second and third generationsthemselves are not shown in FIG. 15.

In the following description of the present invention, an encodingprocess carried out by the transcoder of the first generation isreferred to as an encoding process of the first generation and anencoding process carried out by the transcoder of the second generationconnected after the transcoder of the first generation is referred to asan encoding process of the second generation. Likewise, an encodingprocess carried out by the transcoder of the third generation connectedafter the transcoder of the second generation is referred to as anencoding process of the third generation and an encoding process carriedout by the fourth transcoder connected after the transcoder of the thirdgeneration, that is, the transcoder 101 shown in FIG. 15, is referred toas an encoding process of the fourth generation. In addition, encodingparameters used as well as obtained as a result of the encoding processof the first generation are referred to as encoding parameters of thefirst generation and encoding parameters used as well as obtained as aresult of the encoding process of the second generation are referred toas encoding parameters of the second generation. Similarly, encodingparameters used as well as obtained as a result of the encoding processof the third generation are referred to as encoding parameters of thethird generation and encoding parameters used as well as obtained as aresult of the encoding process of the fourth generation are referred toas encoding parameters of the fourth generation or the current encodingparameters.

First, an encoded video-bitstream ST(3rd) of the third generationgenerated and supplied by the transcoder of the third generation to thetranscoder 101 of the fourth generation shown in FIG. 15 is explained.The encoded video bitstream ST(3rd) of the third generation is anencoded video bitstream obtained as a result of the encoding process ofthe third generation carried out by the third transcoder provided at astage preceding the transcoder 101 of the fourth generation. In theencoded video bitstream ST(3rd) of the third generation resulting fromthe encoding process of the third generation, the encoding parameters ofthe third generation generated in the encoding process of the thirdgeneration are described as a sequence_header( ) function, asequence_extension( ) function, a group_of_pictures_header( ) function,a picture_header( ) function, a picture_coding_extension( ) function, apicture_data( ) function, a slice( ) function and a macroblock( )function on a sequence layer, a GOP layer, a picture layer, a slicelayer and a macroblock layer of the encoded video bitstream ST of thethird generation, respectively. The fact that the encoding parameters ofthe third generation used in the encoding process of the thirdgeneration are described in the encoded video bitstream ST of the thirdgeneration resulting from the encoding process of the third generationconforms to MPEG2 standard and does not reveal any novelty whatsoever.

A point unique to the transcoder 101 provided by the present inventionis not the fact that the encoding parameters of the third generation aredescribed in the encoded video bitstream ST of the third generation, butthe fact that the encoding parameters of the first and secondgenerations obtained as results of the encoding processes of the firstand second generations respectively are included in the encoded videobitstream ST of the third generation. The encoding parameters of thefirst and second generations are described as history_stream( ) in auser_data area of the picture layer of the encoded video bitstream ST ofthe third generation. In the present invention, the history streamdescribed in a user_data area of the picture layer of the encoded videobitstream ST of the third generation is referred to as “historyinformation” and the parameters described as a history stream arereferred to as “history parameters”. In another way of namingparameters, the encoding parameters of the third generation described inthe encoded video bitstream ST of the third generation can also bereferred to as current encoding parameters. In this case, theencoding-parameters of the first and second generations described ashistory_stream( ) in the user-data area of the picture layer of theencoded video bitstream ST of the third generation are referred to as“past encoding parameters” since the encoding processes of the first andsecond generations are each a process carried out in the past if viewedfrom the encoding process of the third generation.

The reason why the encoding parameters of the first and secondgenerations obtained as results of the encoding processes of the firstand second generations respectively are also described in the encodedvideo bitstream ST(3rd) of the third generation in addition to theencoding parameters of the third generation as described above is toavoid deterioration of the picture quality, even if the GOP structureand the bit rate of encoding stream are changed repeatedly intranscoding processes. For example, a picture may be encoded as aP-picture in the encoding process of the first generation and, in orderto change the GOP structure of the encoded video bitstream of the firstgeneration, the picture is encoded as a B-picture in the encodingprocess of the second generation. In order to further change the GOPstructure of the encoded video bitstream of the second generation, thepicture is again encoded as a P-picture in the encoding process of thethird generation. Since conventional encoding and decoding processesbased on the MPEG standard are not 100% reverse processed, the picturequality deteriorates each time encoding and decoding processes arecarried out as is generally known. In such a case, encoding parameterssuch as the quantization scale, the motion vector and the predictionmode are not merely re-computed in the encoding process of the thirdgeneration. Instead, the encoding parameters such as the quantizationscale, the motion vector and the prediction mode generated in theencoding process of the first generation are re-utilized. The encodingparameters such as the quantization scale, the motion vector and theprediction mode newly generated in the encoding process of the firstgeneration obviously each have a precision higher than the counterpartencoding parameters newly generated in the encoding process of the thirdgeneration. Thus, by re-utilizing the encoding parameters generated inthe encoding process of the first generation, it is possible to lowerthe degree to which the picture quality deteriorates even if theencoding and decoding processes are carried out repeatedly.

The processing according to the present invention described above isexemplified by explaining the decoding and encoding processes carriedout by the transcoder 101 of the fourth generation shown in FIG. 15 indetail. The video decoding apparatus 102 decodes an encoded video signalincluded in the encoded video bitstream ST(3rd) of the third generationby using the encoding parameters of the third generation to generatedecoded base-band digital video data. In addition, the video decodingapparatus 102 also decodes the encoding parameters of the first andsecond generations described as a history stream in the user-data areaof the picture layer of the encoded video bitstream ST(3rd) of the thirdgeneration. The configuration and the operation of the video decodingapparatus 102 are described in detail by referring to FIG. 16 asfollows.

FIG. 16 is a diagram showing a detailed configuration of the videodecoding apparatus 102. As shown in the figure, the video decodingapparatus 102 comprises a buffer 81 for buffering a supplied encodedbitstream, a variable-length decoding circuit 112 for carrying out avariable-length decoding process on the encoded bitstream, aninverse-quantization circuit 83 for carrying out inverse quantization ondata completing the variable-length decoding process in accordance witha quantization scale supplied by the variable-length decoding circuit112, an IDCT circuit 84 for carrying out inverse discrete cosinetransformation on DCT coefficients completing the inverse quantization,a processor 85 for carrying out motion compensation processing, aframe-memory unit 86 and a motion-compensation circuit 87.

In order to decode the encoded video bitstream ST(3rd) of the thirdgeneration, the variable-length decoding circuit 112 extracts theencoding parameters of the third generation described on the picturelayer, the slice layer and the macroblock layers of the encoded videobitstream ST(3rd) of the third generation. Typically, the encodingparameters of the third generation extracted by the variable-lengthdecoding circuit 112 include picture_coding_type representing the typeof the picture, quantiser_scale_code representing the size of thequantization-scale step, macrobloc_type representing the predictionmode, motion_vector representing the motion vector,frame/field_motion_type indicating a frame prediction mode or a fieldprediction mode and dct_type indicating a frame-DCT mode or a field-DCTmode. The quantiser_scale_code encoding parameter is supplied to theinverse-quantization circuit 83. On the other hand, the rest of theencoding parameters such as picture_coding_type, quantiser_scale_code,macroblock_type, motion_vector, frame/field_motion_type and dct_type aresupplied to the motion-compensation circuit 87.

The variable-length decoding circuit 112 extracts not only the abovementioned encoding parameters required for decoding the encoded videobitstream ST(3rd) of the third generation, but all other encodingparameters of the third generation to be transmitted to a transcoder ofthe fifth generation connected behind the transcoder 101 shown in FIG.15 as history information of the third generation from the sequencelayer, the GOP layer, the picture layer, the slice layer and themacroblock layer of the encoded video bitstream ST(3rd) of the thirdgeneration. It is needless to say that the above encoding parameters ofthe third generation such as picture_coding_type, quantiser_scale_code,macrobloc_type, motion_vector, frame/field_motion_type and dct_type usedin the process of the third generation as described above are alsoincluded in the history information of the third generation. Theoperator or the host computer determines in advance what encodingparameters are to be extracted as history information in accordance withthe transmission capacity.

In addition, the variable-length decoding circuit 112 also extracts userdata described in the user-data area of the picture layer of the encodedvideo bitstream ST(3rd) of the third generation, supplying the user datato the history decoding apparatus 104.

The history decoding apparatus 104 extracts the encoding parameters ofthe first and second generations described as history information fromthe user data extracted from the picture layer of the encoded videobitstream ST(3rd) of the third generation. To put it concretely, byanalyzing the syntax of the user data received from the variable-lengthdecoding circuit 112, the history decoding apparatus 104 is capable ofdetecting unique History_Data_Id described in the user data and using itfor extracting converted_history_stream( ). Then, by fetching 1-bitmarker bits (marker_bit) inserted into converted_history_stream( ) atpredetermined intervals, the history decoding apparatus 104 is capableof acquiring history_stream( ). By analyzing the syntax ofhistory_stream( ), the history decoding apparatus 104 is capable ofextracting the encoding parameters of the first and second generationsrecorded in history_stream( ). The configuration and the operation ofthe history decoding apparatus 104 will be described in detail later.

In order to eventually supply the encoding parameters of the first,second and third generations to the video encoding apparatus 106 forcarrying out the encoding process of the fourth generation, the historyinformation multiplexing apparatus 103 multiplexes the encodingparameters of the first, second and third generations in base-band videodata decoded by the video decoding apparatus 102. The historyinformation multiplexing apparatus 103 receives the base-band video datafrom the video decoding apparatus 102, the encoding parameters of thethird generation from the variable-length decoding circuit 112 employedin the video decoding apparatus 102 and the encoding parameters of thefirst as well as second generations from the history decoding apparatus104, multiplexing the encoding parameters of the first, second and thirdgenerations in base-band video data. The base-band video data with theencoding parameters of the first, second and third generationsmultiplexed therein is then supplied to the history informationseparating apparatus 105.

Next, a technique of multiplexing the encoding parameters of the first,second and third generations in the base-band video data is explained byreferring to FIGS. 17 and 18. FIG. 17 is a diagram showing a macroblockcomposed of a luminance-signal portion and a color-difference-signalportion with the portions each comprising 16 pixels 16 pixels as definedin accordance with MPEG standard. One of the portions comprising 16pixels 16 pixels is composed of 4 sub-blocks Y[0], Y[1], Y[2] and Y[3]for the luminance signal whereas the other portion is composed of 4sub-blocks Cr[0], Cr[1], Cb[0] and Cb[1] for the color-differencesignal. The sub-blocks Y[0], Y[1], Y[2] and Y[3] and the sub-blocksCr[0], Cr[1], Cb[0] and Cb[1] each comprise 8 pixels 8 pixels.

FIG. 18 is a diagram showing a format of video data. Defined inaccordance with an RDT601 recommended by ITU, the format represents theso-called D1 format used in the broadcasting industry. Since the D1format is standardized as a format for transmitting video data, 1 pixelof video data is expressed by 10 bits.

Base-band video data decoded in conformity with the MPEG standard is 8bits in length. In the transcoder provided by the present invention,base-band video data decoded in conformity with the MPEG standard istransmitted by using the 8 high-order bits D9 to D2 of the 10-bit D1format as shown in FIG. 18. Therefore, the 8-bit decoded video dataleaves 2 low-order bits D1 and D2 unallocated in the D1 format. Thetranscoder provided by the present invention utilizes an unallocatedarea comprising these unallocated bits for transmitting historyinformation.

The data block shown in FIG. 18 is a data block for transmitting a pixelin the 8 sub-blocks of a macroblock. Since each sub-block actuallycomprises 64 (=8 8) pixels as described above, 64 data blocks each shownin FIG. 18 are required to transmit a volume of data of macroblockcomprising the 8 sub-blocks. As described above, the macroblock for theluminance and color-difference signals comprises 8 sub-blocks eachcomposed of 64 (=8 8) pixels. Thus, the macroblock for the luminance andcolor-difference signals comprises 8 64 pixels=512 pixels. Since eachpixel leaves 2 bits unallocated as described above, the macroblock forthe luminance and color-difference signals has 512 pixels 2 unallocatedbits/pixel=1,024 unallocated bits. By the way, history information ofone generation is 256 bits in length. Thus, history information of four(=1,024/256) previous generations can be superposed on the macroblock ofvideo data for the luminance and color-difference signals. In theexample shown in FIG. 18, history information of the first, second andthird generations is superposed on the one macroblock of video data byutilizing the 2 low-order bits of D1 and D0.

The history information separating apparatus 105 extracts base-bandvideo data from the 8 high-order bits of data transmitted thereto as theD1 format and history information from the 2 low-order bits. In theexample shown in FIG. 15, the history information separating apparatus105 extracts base-band video data from transmitted data, supplying thebase-band video data to the video encoding apparatus 106. At the sametime, the history information separating apparatus 105 extracts historyinformation comprising encoding parameters of the first, second andthird generations from the transmitted data, supplying the historyinformation to the signal encoding circuit 106 and the history encodingapparatus 107.

The video encoding apparatus 106 encodes the base-band video datasupplied thereto by the history information separating apparatus 105into a bitstream having a GOP structure and a bit rate specified by theoperator or the host computer. It should be noted that changing the GOPstructure means changing the number of pictures included in the GOP,changing the number of P-pictures existing between two consecutiveI-pictures or changing the number of B-pictures existing between twoconsecutive I-pictures or between an I-picture and a P-picture.

In the embodiment shown in FIG. 15, the supplied base-band video dataincludes history information of the encoding parameters of the first,second and third generations superposed thereon. Thus, the videoencoding apparatus 106 is capable of carrying out an encoding process ofthe fourth generation by selective re-utilization of these pieces ofhistory information so as to lower the degree to which the picturequality deteriorates.

FIG. 19 is a concrete diagram showing the configuration of an encoder121 employed in the video encoding apparatus 106. As shown in thefigure, the encoder 121 comprises a motion-vector detecting circuit 50,a prediction mode switch circuit 52, a processor 53, a DCT switchcircuit 55, a DCT circuit 56, a quantization circuit 57, avariable-length coding circuit 58, a transmission buffer 59, aninverse-quantization circuit 60, an inverse-DCT circuit 61, a processor62, a frame memory unit 63 and a motion compensation 64. The functionsof these circuits are all but the same as those employed in the encoder18 shown in FIG. 7, making it unnecessary to repeat their explanation.The following description thus focuses on differences between theencoder 121 and the encoder 18 shown in FIG. 7.

The encoder 121 also includes a controller 70 for controlling theoperations and the functions of the other aforementioned componentscomposing the encoder 121. The controller 70 receives an instructionspecifying a GOP structure from the operator or the host computer,determining the types of pictures constituting the GOP structure. Inaddition, the controller 70 also receives information on a target bitrate from the operator or the host computer, controlling thequantization circuit 57 so as to set the bit rate output by the encoder121 at the specified target bit rate.

Furthermore, the controller 70 also receives history information of aplurality of generations output by the history information separatingapparatus 105, encoding a referenced picture by reutilization of thishistory information. The functions of the controller 70 are described indetail as follows.

First, the controller 70 forms a judgment as to whether or not the typeof a referenced picture determined from the GOP structure specified bythe operator or the host computer matches with the picture type includedin the history information. That is to say, the controller 70 forms ajudgment as to whether or not the referenced picture has been encoded inthe past in the same picture type as the specified picture type.

The formation of the judgment described above can be exemplified byusing the example shown in FIG. 15. The controller 70 forms a judgmentas to whether or not the picture type assigned to the referenced picturein the encoding process of the fourth generation is the same type asthat of the referenced picture in the encoding process of the firstgeneration, the type of the referenced picture in the encoding processof the second generation or the type of the referenced picture in theencoding process of the third generation.

If the result of the judgment indicates that the picture type assignedto the referenced picture in the encoding process of the fourthgeneration is not the same type as that of the referenced picture in anencoding process of any previous generation, the controller 70 carriesout a normal encoding process. This result of the judgment implies thatthis referenced picture was never subjected to the encoding processes ofthe first, second and third generations previously in the picture typeassigned to the referenced picture in the encoding process of the fourthgeneration. If the result of the judgment indicates that the picturetype assigned to the referenced picture in the encoding process of thefourth generation is the same type as that of the referenced picture inan encoding process of a previous generation, on the other hand, thecontroller 70 carries out a parameter-reuse-encoding-process byreutilization of parameters of the previous generation. This result ofthe judgment implies that this referenced picture was subjected to theencoding process of the first, second or third generation previously inthe picture type assigned to the referenced picture in the encodingprocess of the fourth generation.

First, the normal encoding process carried out by the controller 70 isexplained. In order to allow the controller 70 to make a decision as towhich one of the frame-prediction mode or the field-prediction modeshould be selected, the motion-vector detecting circuit 50 detects aprediction error in the frame prediction mode and a prediction error inthe field prediction mode, supplying the values of the prediction errorsto the controller 70. The controller 70 compares the values with eachother, selecting the prediction mode with the smaller prediction error.The prediction mode switch circuit 52 then carries out signal processingto correspond to the prediction mode selected by the controller 70,supplying a signal obtained as a result of the processing to theprocessing unit 53. With the frame-prediction mode selected, theprediction mode switch circuit 52 carries out signal processing so as tosupply the luminance signal to the processing unit 53 as it is onreceiving the signal, and carries out signal processing of thecolor-difference signal so as to mix odd field lines with even fieldlines as being described earlier by referring to FIG. 8. With thefield-prediction mode selected, on the other hand, the prediction modeswitch circuit 52 carries out signal processing of the luminance signalso as to make the luminance sub-blocks Y[1] and Y[2] comprise odd fieldlines and luminance sub-blocks Y[3] and Y[4] comprise even field lines,and carries out signal processing of the color-difference signal so asto make the four upper lines comprise odd field lines and the four lowerlines comprise even field lines as described earlier by referring toFIG. 9.

In addition, in order to allow the controller 70 to make a decision asto which one of the intra-picture prediction mode, the forwardprediction mode, the backward prediction mode or the forward & backwardprediction mode to be selected, the motion-vector detecting circuit 50generates a prediction error for each of the prediction modes, supplyingthe prediction errors to the controller 70. The controller 70 selects amode having the smallest prediction error as an inter-picture predictionmode from the forward prediction mode, the backward prediction mode orthe forward & backward prediction mode. Then, the controller 70 comparesthe smallest prediction error of the selected inter-picture predictionmode with the prediction error of the intra-picture prediction mode,selecting either the selected inter-picture prediction or theintra-picture prediction mode with a smaller prediction error as aprediction mode. To put it in detail, if the prediction error of theintra-picture prediction mode is found smaller, the intra-pictureprediction mode is established. If the prediction error of theinter-picture prediction mode is found smaller, on the other hand, theselected forward prediction, backward prediction mode or forward &backward prediction mode with the smallest prediction error isestablished. The controller 70 then controls the processor 53 and themotion compensation 64 to operate in the established prediction mode.

Furthermore, in order to allow the controller 70 to make a decision asto which one of the frame-DCT mode or the field-DCT mode to be selected,the DCT mode switch circuit 55 converts data of the four luminancesub-blocks into a signal with a format of the frame-DCT mode comprisingmixed odd and even field lines and a signal with a format of thefield-DCT mode comprising separated odd and even field lines, supplyingthe signals resulting from the conversion to the DCT circuit 56. The DCTcircuit 56 computes an encoding efficiency of DCT processing of thesignal comprising mixed odd and even field lines and an encodingefficiency of DCT processing of the signal comprising separated odd andeven field lines, supplying the computed encoding efficiencies to thecontroller 70. The controller 70 compares the encoding efficiencies witheach other, selecting a DCT mode with a higher efficiency. Thecontroller 70 then controls the DCT mode switch circuit 55 to work inthe selected DCT mode.

The controller 70 also receives a target bit rate representing a desiredbit rate specified by the operator or the host computer and a signalrepresenting the volume of data stored in the transmission buffer 59 orthe size of a residual free area left in the buffer 59, generatingfeedback_q_scale_code for controlling the size of the quantization stepused by the quantization circuit 57 on the basis of the target bit rateand the size of the residual free area left in the buffer 59. Thefeedback_q_scale_code is a control signal generated in accordance withthe size of the residual free area left in the transmission buffer 59 soas to prevent an overflow or an underflow from occurring in the buffer59 and to cause a bitstream to be output from the transmission buffer 59at the target bit rate. To put it concretely, if the volume of databuffered in the transmission buffer 59 becomes small, for example, thesize of the quantization step is reduced so that the number of bits of apicture to be encoded next increases. If the volume of data buffered inthe transmission buffer 59 becomes large, on the other hand, the size ofthe quantization step is increased so that the number of bits of apicture to be encoded next decreases. It should be noted that the sizeof the quantization step is proportional to feedback_q_scale_code. Thatis to say, when feddback_q_scale_code is increased, the size of thequantization step rises. When feddback_q_scale_code is reduced, on theother hand, the size of the quantization step decreases.

Next, the parameter-reuse-encoding process reutilizing encodingparameters which characterizes the transcoder 101 is explained. In orderto make the explanation easy to understand, it is assumed that thereferenced picture was encoded as an I-picture in the encoding processof the first generation, as a P-picture in the encoding process of thesecond generation and as a B-picture in the encoding process of thethird generation, and has to be encoded again as an I-picture in thecurrent encoding process of the fourth generation. In this case, sincethe referenced picture was previously encoded in the encoding process ofthe first generation in the required picture type as the I-pictureassigned to the encoding process of the fourth generation, thecontroller 70 carries out an encoding process by using the encodingparameters of the first generation instead of creating new encodingparameters from the supplied video data. Representatives of suchencoding parameters to be reutilized in the encoding process of thefourth generation include quantiser_scale_code representing the size ofthe quantization-scale step, macrobloc_type representing the predictionmode, motion_vector representing the motion vector,frame/field_motion_type indicating a frame prediction mode or a fieldprediction mode and dct_type indicating a frame-DCT-mode or a field-DCTmode. The controller 70 does not reutilize all encoding parametersreceived as information history. Instead, the controller 70 reutilizesonly encoding parameters judged to be appropriate for reutilization andnewly creates encoding parameters for which previous encoding parametersare not suitable for reutilization.

Next, the parameter-reuse-encoding process reutilizing encodingparameters is explained by focusing on differences from the normalencoding process described earlier. In the normal encoding process, themotion-vector detecting circuit 50 detects a motion vector of areferenced picture. In the parameter-reuse-encoding process reutilizingencoding parameters, on the other hand, the motion-vector detectingcircuit 50 does not detect a motion vector of a referenced picture.Instead, the motion-vector detecting circuit 50 reutilizes motion_vectortransferred as history information of the first generation. The reasonmotion_vector of the first generation is used is explained as follows.Since the base-band video data obtained as a result of a decodingprocess of the encoded bitstream of the third generation has experiencedat least three encoding processes, the picture quality thereof isobviously poor in comparison with the original video data. A motionvector detected from video data with a poor picture quality is notaccurate. To be more specific, a motion vector supplied to thetranscoder 101 of the fourth generation as history information of thefirst generation is certainly has a precision better than a motionvector detected in the encoding process of the fourth generation. Byreutilization of the motion vector received as an encoding parameter ofthe first generation, the picture quality does not deteriorate duringthe encoding process of the fourth generation. The controller 70supplies the motion vector received as the history information of thefirst generation to the motion compensation 64 and the variable-lengthcoding circuit 58 to be used as the motion vector of a referencedpicture being encoded in the encoding process of the fourth generation.

In the normal processing, the motion-vector detecting circuit 50 detectsa prediction error in the frame-prediction mode and a prediction errorin the field-prediction mode in order to select either theframe-prediction mode or the field-prediction mode. In theparameter-reuse-encoding processing based on reutilization of encodingparameters, on the other hand, the motion-vector detecting circuit 50detects neither prediction error in the frame-prediction mode and norprediction error in the field-prediction mode. Instead,frame/field_motion_type received as history information of the firstgeneration to indicate either the frame-prediction mode or thefield-prediction mode is reutilized. This is because the predictionerror of each prediction mode detected in the encoding process of thefirst generation has a precision higher than the prediction error ofeach prediction mode detected in the encoding process of the fourthgeneration. Thus, a prediction mode selected on the basis of predictionerrors each having a high precision will allow a more optimum encodingprocess to be carried out. To put it concretely, the controller 70supplies a control signal representing frame/field_motion_type receivedas history information of the first generation to the prediction modeswitch circuit 52. The control signal drives the prediction mode switchcircuit 52 to carry out signal processing according to reutilizedframe/field_motion_type.

In the normal processing, the motion-vector detecting circuit 50 alsodetects a prediction error in each of the intra-picture prediction mode,the forward prediction mode, the backward prediction mode and theforward & backward prediction mode in order to select one of theseprediction modes. In the processing based on reutilization of encodingparameters, on the other hand, the motion-vector detecting circuit 50detects no prediction errors of these prediction modes. Instead, one ofthe intra-picture prediction mode, the forward prediction mode, thebackward prediction mode and the forward & backward prediction modeindicated by macroblock_type received as history information of thefirst generation is selected. This is because the prediction error ofeach prediction mode detected in the process of the first generation hasa precision higher than the prediction error of each prediction modedetected in the process of the fourth generation. Thus, a predictionmode selected on the basis of prediction errors each having a highprecision will allow a more efficient encoding process to be carriedout. To put it concretely the controller 70 selects a prediction modeindicated by macroblock_type included in the history information of thefirst generation and controls the processor 53 and themotion-compensation 64 to operate in the selected prediction mode.

In the normal encoding process, the DCT mode switch circuit 55 suppliesboth of a signal converted into a format of the frame-DCT mode and asignal converted into a format of the field-DCT mode to the DCT circuit56 for use in comparison of an encoding efficiency in the frame-DCT modewith an encoding efficiency in the field-DCT mode. In the processingbased on reutilization of encoding parameters, on the other hand,neither the signal converted into a format of the frame-DCT mode nor thesignal converted into a format of the field-DCT mode is generated.Instead, only processing in a DCT mode indicated by dct_type included inthe history information of the first generation is carried out. To putit concretely, the controller 70 reutilizes dct_type included in thehistory information of the first generation and controls the DCT-modeswitching circuit 55 to carry out signal processing in accordance with aDCT mode indicated by dct_type.

In the normal encoding process, the controller 70 controls the size ofthe quantization step used in the quantization circuit 57 on the basisof a target bit rate specified by the operator or the host computer andthe size of a residual free area left in the transmission buffer 59. Inthe processing based on reutilization of encoding parameters, on theother hand, the controller 70 controls the size of the quantization stepused in the quantization circuit 57 on the basis of a target bit ratespecified by the operator or the host computer, the size of a residualfree area left in the transmission buffer 59 and a past quantizationscale included in history information. It should be noted that, in thefollowing description, the past quantization scale included in historyinformation is referred to as history_q_scale_code. In a history streamto be described later, the quantization scale is referred to asquantiser_scale_code.

First, the controller 70 generates feedback_q_scale_code representingthe current quantization scale as is the case with the normal encodingprocess. Feedback_q_scale_code is set at such a value determined fromthe size of the residual free area left in the transmission buffer 59that neither overflow nor underflow occurs in the transmission buffer59. Then, history_q_scale_code representing a previous quantizationscale included in the history stream of the first generation is comparedwith feedback_q_scale_code representing the current quantization scaleto determine which quantization scale is greater. It should be notedthat a large quantization scale implies a large quantization step. Iffeedback_q_scale_code representing the current quantization scale isfound greater than history_q_scale_code representing the largest among aplurality of previous quantization scales, the controller 70 suppliesfeedback_q_scale_code representing the current quantization scale to thequantization circuit 57. If history_q_scale_code representing thelargest previous quantization scale is found greater thanfeedback_q_scale_code representing the current quantization scale, onthe other hand, the controller 70 supplies history_q_scale_coderepresenting the largest previous quantization scale to the quantizationcircuit 57. The controller 70 selects the largest among a plurality ofprevious quantization scales included in the history information and thecurrent quantization scale derived from the size of the residual freearea left in the transmission buffer 59. In other words, the controller70 controls the quantization circuit 57 to carry out quantization usingthe largest quantization step among the quantization steps used in thecurrent encoding process (or the encoding process of the fourthgeneration) and the previous encoding processes (or the encodingprocesses of the first, second and third generations). The reason isdescribed as follows.

Assume that the bit rate of a stream generated in the encoding processof the third generation is 4 Mbps and the target bit rate set for theencoder 121 carrying out the encoding process of the fourth generationis 15 Mbps. Such a target bit rate which is higher than the previous bitrate cannot be actually achieved by simply decreasing the quantizationstep. This is because a current encoding process carried out at a smallquantization step on a picture completing a previous encoding processperformed at a large quantization step by no means improves the qualityof the picture. That is to say, a current encoding process carried outat a small quantization step on a picture completing a previous encodingprocess performed at a large quantization step merely increases thenumber of resulting bits, but does not help to improve the quality ofthe picture. Thus, by using the largest quantization step among thequantization steps used in the current encoding process (or the encodingprocess of the fourth generation) and the previous encoding processes(or the encoding processes of the first, second and third generations),the most efficient encoding process can be carried out.

Next, the history decoding apparatus 104 and the history encodingapparatus 107 are explained by referring to FIG. 15. As shown in thefigure, the history decoding apparatus 104 comprises a user-data decoder201, a converter 202 and a history decoder 203. The user-data decoder201 decodes user data supplied by the decoding apparatus 102. Theconverter 202 converts data output by the user-data decoder 201 and thehistory decoder 203 reproduces history information from data output bythe converter 202.

On the other hand, the history encoding apparatus 107 comprises ahistory formatter 211, a converter 212 and a user-data formatter 213.The history formatter 211 formats encoding parameters of the threegenerations supplied thereto by the history information separatingapparatus 105. The converter 212 converts data output by the historyformatter 211 and the user-data formatter 213 formats data output by theconverter 212 into a format of the user data.

The user-data decoder 201 decodes user data supplied by the decodingapparatus 102, supplying a result of the decoding to the converter 202.Details of the user data will be described later. At any rate, the userdata denoted by user_data( ) comprises user_data_start_code anduser_data. According to MPEG specifications, generation of 23consecutive bits of “0” in the user-data is prohibited in order toprevent start_code from being detected incorrectly. Since the historyinformation may include 23 or more consecutive bits of “0”, it isnecessary to process and convert the history information intoconverted_history_stream( ) to be described later by referring to FIG.38. The component that carries out this conversion by insertion of a “1”bit is the converter 212 employed in the history encoding apparatus 107.On the other hand, the converter 202 employed in the history encodingapparatus 104 carries out conversion of deletion of a bit opposite tothe conversion performed by the converter 212 employed in the historyencoding apparatus 107.

The history decoder 203 generates history information from data outputby the converter 202, outputting the information to the historyinformation multiplexing apparatus 103.

On the other hand, the history formatter 211 employed in the historyencoding apparatus 107 converts formats encoding parameters of the threegenerations supplied thereto by the history information separatingapparatus 105 into a format of history information. The format ofhistory information can have a fixed length shown in FIGS. 40 to 46 tobe described later or a variable length shown in FIG. 47 also to bedescribed later.

The history information formatted by the history formatter 211 isconverted by the converter 212 into converted_history_stream( ) toprevent start_code of user_data( ) from being detected incorrectly asdescribed above. That is to say, while history information may include23 or more consecutive bits of “0”, generation of 23 consecutive bits of“0” in user_data is prohibited by MPEG specifications. The converter 212thus converts the history information by insertion of a “1” bit inaccordance with the imposed prohibition.

The user-data formatter 213 adds Data_ID and user_data_stream_code toconverted_history_stream( ) supplied by the converter 212 on the basisof a syntax shown in FIG. 38 to be described later to generate user_datathat can be inserted into video_stream, outputting user_data to theencoding apparatus 106.

FIG. 20 is a block diagram showing a typical configuration of thehistory formatter 211. As shown in the figure, code-language converter301 and a code-language converter 305 receive item data and item-numberdata from the encoding-parameter separating circuit 105. The item datais encoding parameters, that is, encoding parameters transmitted thistime as history information. The item-number data is information usedfor identifying a stream including the encoding parameters. Examples ofthe item-number data are a syntax name and the name of sequence_headerto be described later. The code-language converter 301 converts anencoding parameter supplied thereto into a code language conforming to aspecified syntax, outputting the code to a barrel shifter 302. Thebarrel shifter 302 barrel-shifts the code language supplied thereto by ashift amount corresponding to information supplied thereto by an addressgenerating circuit 306, outputting the shifted code to a switch 303 inbyte units. The switch 303, the contact position of which can be changedover by a bit select signal output by the address generating circuit306, has as many pairs of contact poles as bits supplied by the barrelshifter 302. The switch 303 passes on the code supplied thereto by thebarrel shifter 302 to a RAM unit 304 to be stored at a write addressspecified by the address generating circuit 306. The code languagestored in the RAM unit 304 is read out back from a read addressspecified by the address generating circuit 306 and supplied to theconverter 212 provided at the following stage. If necessary, the coderead out from the RAM unit 304 is again supplied to the RAM unit 304 byway of the switch 303 to be stored therein again.

The code-length converter 305 determines the code length of the encodingparameter from the syntax and the encoding parameter supplied thereto,outputting information on the code length to the address generatingcircuit 306. The address generating circuit 306 generates the shiftamount, the bit select signal, the write address and the read addressdescribed above in accordance with the information on the code lengthreceived from the code-length converter 305. The shift amount, the bitselect signal and the addresses are supplied to the barrel shifter 302,the switch 303 and the RAM unit 304 respectively.

As described above, the history formatter 211 functions as the so-calledvariable-length encoder for carrying out a variable-length encodingprocess on an encoding parameter supplied thereto and for outputting aresult of the variable-length encoding process.

FIG. 22 is a block diagram showing a typical configuration of theconverter 212. In this typical configuration, 8-bit data is read outfrom a read address in a buffer-memory unit 320 provided between thehistory formatter 211 and the converter 212 and supplied to an 8-bitD-type flip-flop (D-FF) circuit 321 to be held therein. The read addressis generated by a controller 326. The 8-bit data read out from theD-type flip-flop circuit 321 is supplied to a stuff circuit 323 and an8-bit D-type flip-flop circuit 322 to be held therein. The 8-bit dataread out from the D-type flip-flop circuit 322 is also supplied to thestuff circuit 323. To put it in detail, the 8-bit data read out from theD-type flip-flop circuit 321 is concatenated with the 8-bit data readout from the D-type flip-flop circuit 322 to form 16-bit parallel datawhich is then supplied to the stuff circuit 323.

The stuff circuit 323 inserts the code “1” into a stuff positionspecified by a signal to produce data with a total of 17 bits which issupplied to a barrel shifter 324. The signal indicating a stuff positionis supplied by the controller 326 and the operation to insert the code“1” is called stuffing.

The barrel shifter 324 barrel-shifts the data supplied thereto by thestuff circuit 323 by a shift amount indicated by a signal received fromthe controller 326, extracting 8-bit data out off the shifted one. Theextracted data is then output to an 8-bit D-type flip-flop circuit 325to be held therein. The data held in the D-type flip-flop circuit 325 isfinally output to the user-data formatter 213 provided at the followingstage by way of a buffer-memory unit 327. That is to say, the data istemporarily stored in the buffer-memory unit 327 provided between theconverter 212 and the user-data formatter 213 at a write addressgenerated by the controller 326.

FIG. 23 is a block diagram showing a typical configuration of the stuffcircuit 323. In this configuration, the 16-bit data received from theD-type flip-flop circuits 321 and 322 is supplied to contact points a ofswitches 331-16 to 331-1. Pieces of data supplied to the contact pointsa of the switches 331-i where i=0 to 15 are also supplied to contactpoints c of the switches 331-i. The switches 331-i where i=1 to 16 areswitches adjacent to the switches 331-i where i=0 to 15 respectively onthe MSB side shown in the figure) of the switches 331-i where i=0 to 15.For example, the thirteenth piece of data from the LSB supplied to thecontact point a of the switch 331-13 on the MSB side of the adjacentswitch 331-12 is also supplied to the contact point c of the switch331-12. At the same time, the fourteenth piece of data from the LSBsupplied to the contact point a of the switch 331-14 on the MSB side ofthe adjacent switch 331-14 is also supplied to the contact point c ofthe switch 331-13.

However, the contact point a of the switch 331-0 provided on the lowerside of the switch 331-1 is open because no switch is provided on thelower side of the switch 331-0 which corresponds to the LSB. Inaddition, the contact point c of the switch 331-16 provided on the upperside of the switch 331-15 is also open because no switch is provided onthe upper side of the switch 331-16 which corresponds to the MSB.

The data “1” is supplied to contact points b of the switches 331-0 to331-16. The decoder 332 changes over one of the switches from 331-0 to331-16 at a stuff position indicated by a staff position signal receivedfrom the controller 326 to the contact point b in order to insert thedata “1” into the stuff position. The switches 331-0 to 331-16 on theLSB side of the switch at the stuff position are changed over to theircontact points c whereas the switches 331 on the MSB side of the switchat the stuff position are changed over to their contact points a.

FIG. 23 shows an example in which the data “1” is inserted into thethirteenth bit from the LSB side. Thus, in this case, the switches 331-0to 331-12 are changed over to their contact points c and the switches331-14 to 331-16 are changed over to their contact points a. The switch331-13 is changed over to the contact point b thereof.

With the configuration described above, the converter 212 shown in FIG.22 converts the 22-bit code into a 23-bit code including the inserteddata “1”, outputting the 23-bit result of the conversion.

FIG. 24 is a diagram showing timing of pieces of data output by avariety of portions of the converter 212 shown in FIG. 22. When thecontroller 326 employed in the converter 212 generates a read addressshown in FIG. 24A synchronously with a clock signal for a byte of data,byte data stored at the read address is read out from the buffer-memoryunit 320 and temporarily held by the D-type flip-flop circuit 321. Then,data of FIG. 24B read out from the D-type flip-flop circuit 321 issupplied to the stuff circuit 323 and the D-type flip-flop circuit 322to be held therein. Data of FIG. 24C read out from the D-type flip-flopcircuit 322 is concatenated with the data of FIG. 24B read out from theD-type flip-flop circuit 321 and data obtained as a result of theconcatenation shown in FIG. 24D is supplied to the stuff circuit 323.

As a result, with timing of a read address A1, the first byte D0 of thedata of FIG. 24B read out from the D-type flip-flop circuit 321 issupplied to the stuff circuit 323 as a first byte of the data shown inFIG. 24D. Then, with timing of a read address A1, the second byte D1 ofthe data of FIG. 24B read out from the D-type flip-flop circuit 321 andconcatenated with the first byte D0 of the data of FIG. 24C read outfrom the D-type flip-flop circuit 322 is supplied to the stuff circuit322 as a second two bytes of the data shown in FIG. 24D. Subsequently,with timing of a read address A3, the third byte D2 of the data of FIG.24B read out from the D-type flip-flop circuit 321 and concatenated withthe second byte D1 of the data of FIG. 24C read out from the D-typeflip-flop circuit 322 is supplied to the stuff circuit 322 as a thirdtwo bytes of the data shown in FIG. 24D and so on.

The stuff circuit 323 receives a signal of FIG. 24E indicating a stuffposition, into which the data “1” is to be inserted, from the controller326. The decoder 332 employed in the stuff circuit 323 changes over oneof the switches 331-0 to 331-16 at the stuff position to the contactpoint b thereof. The switches 331 on the LSB side of the switch at thestuff position are changed over to their contact points c whereas theswitches 331 on the MSB side of the switch at the stuff position arechanged over to their contact points a. As a result, the stuff circuit323 inserts the data “1” into the stuff position signal, outputting datawith the inserted data “1” shown in FIG. 24F.

The barrel shifter 324 barrel-shifts the data supplied thereto by thestuff circuit 323 by a shift amount indicated by a shift signal of FIG.24G received from the controller 326, outputting a shifted signal shownin FIG. 24H. The shifted signal is held temporarily in the D-typeflip-flop circuit 325 before being output to a later stage as shown inFIG. 24I.

The data output from the D-type flip-flop circuit 325 includes the data“1” inserted into a position following to 22-bit data. Thus, the numberof consecutive 0 bits never exceeds 22 even if all bits between the data“1” and the next data “1” are “0”.

FIG. 25 is a block diagram showing a typical configuration of theconverter 202. The fact that components ranging from a D-type flip-flopcircuit 341 to a controller 346 are employed in the converter 202 as thecomponents ranging from the D-type flip-flop circuit 321 to thecontroller 326 are employed in the converter 212 shown in FIG. 22indicates that the configuration of the former is basically the same asthe configuration of the latter. The converter 202 is different from theconverter 212 in that, in the case of the former, a delete circuit 343is employed in place of the stuff circuit 323 of the latter. Otherwise,the configuration of the converter 202 is the same as that of theconverter 212 shown in FIG. 22.

The delete circuit 343 employed in the converter 202 deletes a bit at adelete position indicated by a signal output by the controller 346. Thedelete position corresponds to the stuff position, into which the stuffcircuit 323 shown in FIG. 23 inserts the data “1”.

The remaining operations of the converter 202 are the same as thosecarried out by the converter 212 shown in FIG. 22.

FIG. 26 is a block diagram showing a typical configuration of the deletecircuit 343. In this configuration, the 15 bits on LSB side of which16-bit data received from the D-type flip-flop circuits 341 and 342 aresupplied to contact points a of switches 351-0 to 351-14. Pieces of datasupplied to the contact points a of the switches 351-i where i=1 to 14are also supplied to contact points b of the switches 351-i where i=0 to13. The switches 351-i where i=1 to 14 are switches adjacent to theswitches 351-i where i=0 to 13 respectively on the MSB side (or theupper side shown in the figure) of the switches 351-i where i=0 to 13.For example, the thirteenth piece of data from the LSB supplied to thecontact point a of the switch 351-13 on the MSB side of the adjacentswitch 351-12 is also supplied to the contact, point b of the switch351-12. At the same time, the 14th piece of data from the LSB suppliedto the contact point a of the switch 351-14 on the MSB side of theadjacent switch 351-13 is also supplied to the contact point c of theswitch 351-13. The decoder 352 deletes a bit at a delete positionindicated by a signal output by the controller 346, outputting theremaining 15-bit data excluding the deleted bit.

FIG. 26 shows a state in which the thirteenth input bit from the LSB(input bit 12) is deleted. Thus, in this case, the switches 351-0 to351-11 are changed over to their contact points a to output 12 inputbits from the LSB (bit 0) to the twelfth bit (bit 11) as they are. Onthe other hand, the switches 351-12 to 351-14 are changed over to theircontact points b to pass on the fourteenth to sixteenth input bits(input bits 13 to 15) as the thirteenth to fifteenth output bits (outputbits 12 to 14) respectively. In this state, the thirteenth input bitfrom the LSB (input bit 12) is connected to none of the output lines.

16-bit data is supplied to the stuff circuit 323 shown in FIG. 23 andthe delete circuit 343 shown in FIG. 26. This is because the datasupplied to the stuff circuit 323 is a result of concatenation of piecesof data output by the 8-bit D-type flip-flop circuits 321 and 322employed in the converter 212 shown in FIG. 22. By the same token, thedata supplied to the delete circuit 343 is a result of concatenation ofpieces of data output by the 8-bit D-type flip-flop circuits 341 and 342employed in the converter 202 shown in FIG. 25. The barrel shifter 324employed in the converter 212 shown in FIG. 22 barrel-shifts the 17-bitdata supplied thereto by the stuff circuit 323 by a shift amountindicated by a signal received from the controller 326, finallyextracting data of typically 8 bits out off the shifted one. Likewise,the barrel shifter 344 employed in the converter 202 shown in FIG. 25barrel-shifts the 15-bit data supplied thereto by the stuff circuit 324by a shift amount indicated by a signal received from the controller346, finally extracting data of 8 bits out off the shifted one.

FIG. 21 is a block diagram showing a typical configuration of thehistory decoder 203 for decoding data completing the history formattingprocess in the converter 202. Data of an encoding parameter supplied tothe history decoder 203 by the converter 202 is fed to a RAM unit 311,to be stored therein at a write address specified by an addressgenerating circuit 315. The address generating circuit 315 also outputsa read address with predetermined timing to the RAM unit 311. At thattime, data stored at the read address in the RAM unit 311 is output to abarrel shifter 312. The barrel shifter 312 barrel-shifts the datasupplied thereto by a shift amount corresponding to information suppliedthereto by the address generating circuit 315, outputting the shifteddata to inverse-code-length converters 313 and 314.

The inverse-code-length converters 313 and 314 also receive the name ofa syntax of a stream including the encoding parameters from theconverter 202. The inverse code-length converter 313 determines the codelength of the encoding parameter from the data or the code languagesupplied thereto in accordance with the syntax, outputting informationon the code length to the address generating circuit 315.

On the other hand, the inverse-code-length converter 314 decodes orreversely encodes the data supplied by the barrel shifter 312 on thebasis of the syntax, outputting a result of the decoding process to theencoding-parameter multiplexing apparatus 103.

In addition, the inverse-code-length converter 314 also extractsinformation required for identifying what code language is included,that is, information required for determining a delimiter of a code,outputting the information to the address generating circuit 315. Theaddress generating circuit 315 generates write and read addresses and ashift amount based on this information and the code length received fromthe inverse-code-length converter 313, outputting the write/readaddresses to the RAM unit 311 and the shift amount to the barrelregister 312.

FIG. 27 is a block diagram showing another typical configuration of theconverter 212. A counter 361 employed in this configuration counts thenumber of consecutive 0 bits of data supplied thereto, outputting theresult of counting to the controller 326. When the number of consecutive0-bit of data supplied to the counter 361 reaches 22, the controller 326outputs a signal representing a stuff position to the stuff circuit 323.At the same time, the controller 326 resets the counter 361, allowingthe counter 361 to start counting the number of consecutive 0-bit againfrom 0. The rest of the configuration and the operation are the same asthose of the converter 212 shown in FIG. 22.

FIG. 28 is a block diagram showing another typical configuration of theconverter 202. A counter 371 employed in this configuration counts thenumber of consecutive 0-bit of data supplied thereto, outputting theresult of counting to the controller 346. When the number of consecutive0-bit of data supplied to the counter 371 reaches 22, the controller 346outputs a signal representing a delete position to the delete circuit343. At the same time, the controller 346 resets the counter 371,allowing the counter 371 to start counting the number of consecutive0-bit again from 0. The rest of the configuration and the operation arethe same as those of the converter 202 shown in FIG. 25.

As described above, in the typical configurations shown in FIGS. 27 and28, the data “1” is inserted as a marker bit and deleted respectivelywhen a predetermined pattern comprising a predetermined number ofconsecutive 0-bit is detected by the counter. The typical configurationsshown in FIGS. 27 and 28 allow processing to be carried out with ahigher degree of efficiency than the configurations shown in FIGS. 22and 25 respectively.

FIG. 29 is a block diagram showing a typical configuration of theuser-data formatter 213. In this configuration, when a controller 383outputs a read address to a buffer memory not shown provided between theconverter 212 and the user-data formatter 213, data is output from theread address and supplied to a contact point a of a switch 382 employedin the user-data formatter 213. It should be noted that the buffermemory itself is not shown in the figure. In a ROM unit 381, datarequired for generating user_data( ) such as a user-data start code anda data ID is stored. A controller 313 changes over the switch 382 to thecontact point a or a contact point b with predetermined timing in orderto allow the switch 382 to select the data stored in the ROM unit 381 ordata supplied by the converter 212 and pass on the selected data. Inthis way, data with a format of user_data( ) is output to the encodingapparatus 106.

It is worth noting that the user-data decoder 201 can be implemented byoutputting input data by way of a switch for deleting inserted data readout from a ROM unit like the ROM unit 381 employed in the user-dataformatter 213 shown in FIG. 29. The configuration of the user-datadecoder 201 is shown in none of the figures.

FIG. 30 is a block diagram showing the state of an implementation inwhich a plurality of transcoders 101-1 to 101-N are connected in seriesfor the use of video editing studio service. The encoding-parametermultiplexing apparatuses 103-i employed in the transcoders 101-i wherei=1 to N each write most recent encoding parameters used by itself overa region for storing least recent encoding parameters in an area usedfor recording encoding parameters. As a result, base-band picture dataincludes encoding parameters or generation history information of thefour most recent generations associated with the macroblocks of thepicture data.

The variable-length-encoding circuit 58 employed in the encoder 121-i ofFIG. 19 employed in the encoding apparatus 106-i encodes video datareceived from the quantization circuit 57 on the basis of currentencoding parameters received from the encoding-parameter separatingcircuit 105-i. As a result, the current encoding parameters aremultiplexed typically in picture_header( ) included in a bitstreamgenerated by the variable-length-encoding circuit 58.

In addition, the variable-length-encoding circuit 58 also multiplexesuser data, which includes generation history information and is receivedfrom the history encoding apparatus 107-i, into the output bitstream.This multiplexing process is not the embedded processing like the oneshown in FIG. 18, but multiplexing of the user data into the bitstream.Then, the bitstream output by the encoding apparatus 106-i is suppliedto the transcoder 101-(i+1) at the following stage by way of the SDTI108-i.

The configurations of the transcoders 101-i and 101-(i+1) are the sameas the one shown in FIG. 15. The processing carried out by them can thusbe explained by referring to FIG. 15. If it is desired to change thecurrent picture type from the I-picture to the P- or B-picture in anencoding operation using the history of actual encoding parameters, thehistory of previous encoding parameters is searched for those of a P- orB-picture used in the previous. If a history of a P or B-picture isfound in the history, its parameters including a motion vector are usedto change the picture type. If a history of a P- or B-picture is notfound in the history, on the other hand, the modification of the picturetype without motion detection is given up. It is needless to say thatthe picture type can be changed even if an encoding parameter of a P- orB-picture is not found in the history provided that motion detection iscarried out.

In the format shown in FIG. 18, encoding parameters of four generationsare embedded in picture data. As an alternative, parameters for each ofthe I-, P- and B-pictures can also be embedded in a format like oneshown in FIG. 31. In the example shown in FIG. 31, encoding parametersor picture history information of one generation are recorded for eachpicture type in an operation to encode the same macroblocks accompanyingchanges in picture type occurring in the previous. In this case, thedecoder 111 shown in FIG. 16 outputs encoding parameters of onegeneration for the I-, P- and B-pictures to be supplied to the encoder121 shown in FIG. 19 instead of encoding parameters of most recent; thefirst, second and third preceding generations.

In addition, since the area of Cb[1][x] and Cr[1] [x] is not used, thepresent invention can also be applied to picture data of a 4:2:0 formatwhich does not use the area of Cb[1][x] and Cr[1] [x]. In the case ofthis example, the decoding apparatus 102 fetches encoding parameters inthe course of decoding and identifies the picture type. The decodingapparatus 102 writes or multiplexes the encoding parameters into alocation corresponding to the picture type of the picture signal andoutputs the multiplexed picture signal to the encoding-parameterseparating apparatus 105. The encoding-parameter separating apparatus105 separates the encoding parameters from the picture data and, byusing the separated encoding parameters, the encoding-parameterseparating apparatus 105 is capable of carrying out apost-decoding-encoding process while changing the picture type by takinga picture type to be changed and previous encoding parameters suppliedthereto into consideration.

The transcoder 101 has another operation which is different from theparameter-reuse-encoding process to deter mine a changeable picture typein only case of the controller 70 does not allow the motion vectordetecting circuits to operate.

The other operation is explained by referring to a flowchart shown inFIG. 32. As shown in FIG. 32, the flowchart begins with a step S1 atwhich encoding parameters or picture history information of onegeneration for each picture type are supplied to the controller 70 ofthe encoder 121. The flow of the processing then goes on to a step S2 atwhich the encoding-parameter separating apparatus 105 forms a judgmentas to whether or not picture history information includes encodingparameters used in a change to a B-picture. If the picture historyinformation includes encoding parameters used in a change to aB-picture, the flow of the processing proceeds to a step S3.

At the step S3, the econtroller 70 forms a judgment as to whether or notpicture history information includes encoding parameters used in achange to a P-picture. If the picture history information includesencoding parameters used in a change to a P-picture, the flow of theprocessing proceeds to a step S4.

At the step S4, the controller 70 determines that changeable picturetypes are the I-, P- and B-pictures. If the outcome of the judgmentformed at the step S3 indicates that the picture history informationdoes not include encoding parameters used in a change to a P-picture, onthe other hand, the flow of the processing proceeds to a step S5.

At the step S5, the controller 70 determines that changeable picturetypes are the I- and B-pictures. In addition, the controller 70determines that a pseudo-change to a P-picture is also possible bycarrying out special processing using only a forward-prediction vectorand no backward-prediction vector included in history information of theB-picture. If the outcome of the judgment formed at the step S2indicates that the picture history information does not include encodingparameters used in a change to a B-picture, on the other hand, the flowof the processing proceeds to a step S6.

At the step S6, the controller 70 forms a judgment as to whether or notpicture history information includes encoding parameters used in achange to a P-picture. If the picture history information includesencoding parameters used in a change to a P-picture, the flow of theprocessing proceeds to a step S7.

At the step S7, the controller 70 determines that changeable picturetypes are the I- and P-pictures. In addition, the encoding-parameterseparating apparatus 105 determines that a change to a B-picture is alsopossible by carrying out special processing using only aforward-prediction vector and no backward-prediction vector included inhistory information of the P-picture.

If the outcome of the judgment formed at the step S6 indicates that thepicture history information does not include encoding parameters used ina change to a P-picture, on the other hand, the flow of the processingproceeds to a step S8. At the step S8, the controller 70 determines thatthe only changeable picture type is the I-picture because there is nomotion vector. An I-picture can not be changed to any other picture typethan the I-picture.

After completing the steps S4, S5, S7 or S8, the flow of the processinggoes on to a step S9 at which the controller 70 notifies the user of thechangeable picture types on a display unit which is shown in none of thefigures.

FIG. 33 is a diagram showing examples of changes in picture type. Whenthe picture type is changed, the number of frames composing a GOPstructure is changed. To put it in detail, in these examples, a long GOPstructure is changed to a short GOP structure of the second generation.And then, the GOP structure of athe second generation is changed back toa long GOP at a third generation. The long GOP structure has an N=15 andan M=3, where N is the number of frames constituting the GOP and M isthe period of the appearance of the P-picture expressed in terms offrames. On the other hand, the short GOP has an N=1 and an M=1 where Mis the period of the appearance of the I-picture expressed in terms offrames. It should be noted that a dashed line shown in the figurerepresents a boundary between two adjacent GOPs.

When the GOP structure of the first generation is changed to the GOPstructure of the second generation, the picture types of all the framescan be changed to the I-picture as is obvious from the explanation ofthe processing to determine changeable picture types given above. Whenthese picture types are changed, all motion vectors which were processedwhen the source video signal was encoded in the first generation aresaved or left. Then, when the short GOP structure is changed back to thelong GOP structure at the third generation. That is, even if picturetypes are changed, the motion vector for each type which was saved whenthe source video signal was encoded at the first generation isre-utilized, allowing a change back to the long GOP structure to be madewith deterioration of the picture quality avoided.

FIG. 34 is a diagram showing another examples of changes in picturetype. In the case of these examples, changes are made from a long GOPstructure with N=14 and M=2 to a short GOP structure with N=2 and M=2 atthe second generation and then to a short GOP structure with N=1 and M=1and finally to a random GOP with an undetermined frame count N at thefourth generation.

Also in these examples, a motion vector for each picture type which wasprocessed when the source video signal was encoded as the firstgeneration is saved until the fourth generation. As a result, byre-utilizing the saved encoding parameters, deterioration of the picturequality can be reduced to a minimum even if the picture types arechanged in a complicated manner as shown in FIG. 34. In addition, if thequantization scale of the saved encoding parameters is utilizedeffectively, an encoding process which entails only little deteriorationof the picture quality can be implemented.

The re-utilization of the quantization scale is explained by referringto FIG. 35. FIG. 35 is a diagram showing a case in which a certainreference frame is always encoded with an I-picture from a firstgeneration to a fourth generation. Only the bit rate is changed from 4Mbps for the first generation to 18 Mbps for the second generation andthen to 50 Mbps for the third generation and finally back to 4 Mbps forthe fourth generation.

When a bit rate of 4 Mbps of bitstream generated at first generation ischanged to a bit rate of 18 Mbps at second generation, the picturequality is not improved even a post-decoding-encoding process is carriedout at a fine quantization scale accompanying the increase in bit rate.This is because data quantized in the previous at a coarse quantizationstep is not restored. Thus, quantization at a fine quantization stepaccompanying a raise in bit rate in the course of processing as shown inFIG. 35 merely increases the amount of information and does not lead toan improvement of the picture quality. For this reason, if control isexecuted to sustain a coarsest or largest quantization scale used in theprevious, the encoding process can be implemented least wastefully andmost efficiently.

As described above, when the bit rate is changed, by making use of theprevious history of the quantization scale, the encoding process can beimplemented most effectively.

This quantization control processing is explained by referring to aflowchart shown in FIG. 36. As shown in the figure, the flowchart beginswith a step S11 at which controller 70 forms a judgment as to whether ornot input picture history information includes an encoding parameter ofa picture type to be changed from now on. If the outcome of the judgmentindicates that the input picture history information includes anencoding parameter of a picture type to be changed, the flow of theprocessing goes on to a step S12.

At the step S12, the controller 70 extracts the history_q_scale_codefrom the encoding parameters in question for comparison included in thepicture history information.

The flow of the processing then proceeds to a step S13 at which thecontroller 70 calculates a candidate value of the feedback_q_scale_codeobased on a data fullness of the transmission buffer 59.

The flow of the processing then proceeds to a step S14 at which thecontroller 70 forms a judgment as to whether or not history_q_scale_codeis larger or coarser than feedback_q_scale_code. If the outcome of thejudgment indicates that history_q_scale_code is larger or coarser thanfeedback_q_scale_code, the flow of the processing continues to a stepS15.

At the step S15, the controller 70 supplies history_q_scale_code as aquantization scale to the quantization circuit 57 which then carries outa quantization process by using history_q_scale_code.

The flow of the processing then proceeds to a step S16 at which thecontroller 70 forms a judgment as to whether or not all macroblocksincluded in the frame have been quantized. If the outcome of thejudgment indicates that all the macroblocks included in the frame nothave been quantized yet, the flow of the processing goes back to thestep S13 to carry out the pieces of processing of the steps S13 to S16repeatedly till all the macroblocks included in the frame are quantized.

If the outcome of the judgment formed at the step S14 indicates thathistory_q_scale_code is not larger than feedback_q_scale_code, that is,history_q_scale_code is finer than feedback_q_scale_code, on the otherhand, the flow of the processing continues to a step S17.

At the step S17, the controller 70 supplies feeds backfeedback_q_scale_code as a quantization scale to the quantizationcircuit 57 which then carries out a quantization process by usingfeedback_q_scale_code.

If the outcome of the judgment formed at the step S11 indicates that theinput picture history information does not include an encoding parameterof a picture type to be changed, on the other hand, the flow of theprocessing goes on to a step S18.

At the step S18, the quantization circuit 57 receives the candidatevalue of the feedback_q_scale_code from the controller 70.

The flow of the processing then proceeds to a step S19 at which thequantization circuit 57 which carries out a quantization process byusing Q_feedback.

The flow of the processing then proceeds to a step S20 at whichcontroller 70 forms a judgment as to whether or not all macroblocksincluded in the frame have been quantized. If the outcome of thejudgment indicates that all the macroblocks included in the frame nothave been quantized yet, the flow of the processing goes back to thestep S18 to carry out the pieces of processing of the steps S18 to S20repeatedly till all the macroblocks included in the frame are quantized.

The transcoder 101 explained earlier by referring to FIG. 15 suppliesprevious encoding parameters of the first, second and third generationsto the video encoding apparatus 106 by multiplexing these parameters inbase-band video data. In the present invention, however, a technology ofmultiplexing previous encoding parameters in base-band video data is notabsolutely required. For example, previous encoding parameters can betransferred by using a transmission line such as a data transfer busprovided separately from that for the base-band video data as shown inFIG. 37.

The video decoding apparatus 102, the history decoding apparatus 104,the video encoding apparatus 106 and the history encoding apparatus 107shown in FIG. 37 have entirely the same configurations and functions asthe video decoding apparatus 102, the history decoding apparatus 104,the video encoding apparatus 106 and the history encoding apparatus 107respectively which have been described earlier by referring to FIG. 15.

The variable-length decoding circuit 112 employed in the video decodingapparatus 102 extracts encoding parameters of the third generation fromthe sequence layer, the GOP layer, the picture layer, the slice layerand the macroblock layer of the encoded video bitstream ST(3rd) of thethird generation, supplying the parameters to the history encodingapparatus 107 and the controller 70 employed in the video encodingapparatus 106. The history encoding apparatus 107 converts the encodingparameters of the third generation supplied thereto intoconverted_history_stream( ) which can be described in the user-data areaon the picture layer, supplying converted_history_stream( ) to thevariable-length coding circuit 58 employed in the video encodingapparatus 106 as user data.

In addition, the variable-length decoding circuit 112 also extracts userdata (user_data) including previous encoding parameters of the first andsecond generations from the user-data area on the picture layer of theencoded video bitstream ST(3rd) of the third generation, supplying theuser_data to the history decoding apparatus 104 and the variable-lengthcoding circuit 58 employed in the video encoding apparatus 106. Thehistory decoding apparatus 104 extracts the encoding parameters of thefirst and second generations from a history steam of the user data whichis described in the user-data area as converted_history_stream( ),supplying the parameters to the controller 70 employed in the videoencoding apparatus 106.

The controller 70 of the video encoding apparatus 106 controls theencoding process carried out by the video encoding apparatus 106 on thebasis of the encoding parameters of the first and second generationsreceived from the history decoding apparatus 104 and the encodingparameters of the third generation received from the video decodingapparatus 102.

In the meantime, the variable-length coding circuit 58 employed in thevideo encoding apparatus 106 receives the user data (user_data)including encoding parameters of the first and second generations fromthe video decoding apparatus 102 and the user data (user_data) includingencoding parameters of the third generation from the history encodingapparatus 107, describing these pieces of user_data in the user-dataarea on the picture layer of an encoded video bitstream of the fourthgeneration as history information.

FIG. 38 is a diagram showing a syntax used for decoding an MPEG videostream. The decoder decodes an MPEG bit stream in accordance with thissyntax in order to extract a plurality of meaningful data items ormeaningful data elements from the bitstream. In the syntax to beexplained below, a function and a conditional statement are eachrepresented by a string of normal characters whereas a data element isrepresented by a string of bold characters. A data item is described bythe Mnemonic representing the name of the data item. In some cases, theMnemonic also indicates the bit length composing the data item and thetype of the data item.

First of all, functions used in the syntax shown in FIG. 38 areexplained. A next_start_code( ) is a function used for searching abitstream for a start code described in the bitstream. In the syntaxshown in FIG. 38, the next_start_code( ) function is followed by asequence_header( ) function and a sequence_extension( ) function whichare laid out sequentially to indicate that the bitstream includes dataelements defined by the sequence_header( ) and sequence_extension( )functions. Thus, a start code, a kind of data element described at thebeginning of the sequence_header( ) and sequence_extension( ) functions,is found by the next_start_code( ) function from the bitstream in anoperation to decode the bitstream. The start code is then used as areference to further find the sequence_header( ) and sequence_extension() functions and decode data elements defined by the sequence_header( )and sequence_extension( ) functions.

It should be noted that the sequence_header( ) function is a functionused for defining header data of a sequence layer in the MPEG bitstreamwhereas the sequence_extension( ) function is a function used fordefining extension data of a sequence layer in the MPEG bitstream.

A do { } while statement is described after the sequence_extension( )function. The do { } while statement comprises a { } block following ado statement and a while statement following the { } block. Dataelements described by functions in the { } block following the dostatement are extracted from the bitstream as long as a conditiondefined by the while statement is true. That is to say, the do { } whilesyntax defines a decoding process to extract data elements described byfunctions in the { } block following the do statement from the bitstreamas long as the condition defined by the while statement is true.

A nextbits( ) used in the while statement is a function used to comparea bit or a string of bits appearing in the bitstream with a data elementto be decoded next. In the example of the syntax shown in FIG. 38, thenextbits( ) function compares a string of bits appearing in thebitstream with sequence_end_code used for indicating the end of a videosequence. The condition defined by the while statement is the to be trueif the string of bits appearing in the bitstream does not matchsequence_end_code. Thus, the do { } while statement described after thesequence_extension( ) function indicates that data elements defined byfunctions in the { } block following the do statement are described inthe bitstream as long as sequence_end_code used for indicating the endof a video sequence does not appear in the bitstream.

After the data elements defined by the sequence_extension( ) function inthe bitstream, data elements defined by an extension_and_user_data(0)function are described. The extension_and_user_data(0) function is afunction used for defining extension data and user data of the sequencelayer of the MPEG bitstream.

A do { } while statement following the extension_and_user_data(0)function is a function to extract data elements described by functionsin the { } block following the do statement from the bitstream as longas a condition defined by the while statement is true. The nextbits( )functions used in the while statement are functions used to form ajudgment as to whether or not a bit or a string of bits appearing in thebitstream matches picture_start_code or group_start_code start codesrespectively by comparing the string with the start code specified inthe function. If the string of bits appearing in the bitstream matchespicture_start_code or group_start_code, a condition defined by the whilestatement is said to be true. Thus, if picture_start_code orgroup_start_code appears in the bitstream, the codes of data elementsdefined by functions in the { } block following the do statement aredescribed after this start code. Accordingly, by finding a start coderepresented by picture_start_code or group_start_code, it is possible toextract data elements defined by functions in the { } block of the dostatement from the bitstream.

An if-statement described at the beginning of the { } block of the dostatement states a condition “if group_start_code appears in thebitstream.” A true (satisfied) condition stated by the if-statementindicates that data elements defined by a group_of_picture_header(1)function and a extension_and_user_data(1) function are describedsequentially after group_start_code.

The group_of_picture_header(1) function is a function used for definingheader data of a GOP layer of the MPEG bitstream and theextension_and_user_data(1) function is a function used for definingextension data named extension_data and/or user data named user_data ofthe GOP layer of the MPEG bitstream.

Furthermore, in this bitstream, data elements defined by apicture_header( ) function and a picture_coding_extension( ) functionare described after the data elements defined by thegroup_of_picture_header(1) function and the extension_and_user_data(1)function. Of course, if the condition defined by the if-statement is nottrue, the data elements defined by the group_of_picture_header(1)function and the extension_and_user_data(1) function are not described.In this case, the data elements defined by the picture_header( )function and the picture_coding_extension( ) function are describedafter the data elements defined by an extension_and_user_data(0)function.

The picture_header( ) function is a function used for defining headerdata to a picture layer of the MPEG stream and thepicture_coding_extension( ) function is a function used for definingfirst extension data of the picture layer of the MPEG stream.

The next while statement is a function used for defining a condition. Acondition defined by each if-statement described in a { } blockfollowing the condition defined by the while statement is judged to betrue or false as long as the condition defined by the while statement istrue. nextbits( ) functions used in the while statement are functionsfor forming a judgment as to whether a string of bits appearing in thebitstream matches extension_start_code and user_start_code respectively.If the string of bits appearing in the bitstream matchesextension_start_code or user_data_start, a condition defined by thewhile statement is said to be true.

A first if-statement in the { } block following the while statement is afunction for forming a judgment as to whether or not a string of bitsappearing in the bitstream matches extension_start_code. A string ofbits appearing in the bitstream that matches 32-bit extension_start_codeindicates that data elements defined by an extension_data(2) functionare described after extension_start_code in the bitstream.

A second if-statement is a function for forming a judgment as to whetheror not a string of bits appearing in the bitstream matchesuser_data_start_code. If a string of bits appearing in the bitstreammatches 32-bit user_data_start_code, a condition defined by a thirdif-statement is judged to be true or false. The user_data_start_code isa start code used for indicating the beginning of a user-data area ofthe picture layer of the MPEG bitstream.

The third if-statement in the { } block following the while statement isa function for forming a judgment as to whether or not a string of bitsappearing in the bitstream matches History_Data_ID. A string of bitsappearing in the bitstream that matches 8-bit History_Data_ID indicatesthat data elements defined by a converted_history_stream( ) function aredescribed after a code indicated by 8-bit History_Data_ID in theuser-data area of the picture layer of the MPEG bitstream.

A converted_history_stream( ) function is a function used for describinghistory information and history data for transmitting all encodingparameters used in the MPEG encoding process. Details of data elementsdefined by this converted_history_stream( ) function will be describedlater. History_Data_ID is a start code used for indicating the beginningof a description of the history information and the history data in theuser-data area of the picture layer of the MPEG bitstream.

An else statement is syntax indicating a case for a false conditiondefined by the third if-statement. Thus, if data elements defined by aconverted_history_stream( ) function are not described in the user-dataarea of the picture layer of the MPEG bitstream, data elements definedby a user_data( ) function are described.

A picture_data( ) function is a function used for describing dataelements related to a slice layer and a macroblock layer after user dataof the picture layer of the MPEG bitstream. Normally, data elementsdefined by this picture_data( ) function are described after dataelements defined by a user_data( ) function or data elements defined bya converted_history_stream( ) function described in the user-data areaof the picture layer of the bitstream. If neither extension_start_codenor user_data_start_code exists in a bitstream showing data elements ofthe picture layer, however, data elements defined by this picture_data() function are described after data elements defined by apicture_coding_extension( ) function.

After the data elements defined by this picture_data( ) function, dataelements defined by a sequence_header( ) function and asequence_extension( ) function are described sequentially. The dataelements described by the sequence_header( ) function and thesequence_extension( ) function are exactly the same data elements asdefined by a sequence_header( ) function and a sequence_extension( )function described at the beginning of a sequence of the video stream.The reason why the same pieces of data are defined in the stream is toprevent data of the sequence layer from being no longer receivable,thus, preventing a stream from being no longer decodable when receptionof a bitstream is started by a bitstream receiving apparatus in themiddle of a data stream, such as part of bitstream corresponding to apicture layer.

After the data elements defined by the sequence_header( ) function andthe sequence_extension( ) function, that is, at the end of the datastream, 32-bit sequence_end_code used for indicating the end of thesequence is described.

FIG. 39 is a schematic diagram showing an outline of the basicconfiguration of the syntax described so far.

Next, a history stream defined by a converted_history_stream( ) functionis explained.

The converted_history_stream( ) function is a function used forinserting a history stream showing history information into theuser-data area of the picture layer of the MPEG bitstream. It should benoted that the word ‘converted’ means that the stream has completed aconversion process to insert one marker bit for at least every 22 bitsof the history stream composed of history data to be inserted into theuser area in order to avoid start emulation.

The converted_history_stream( ) function is described in either one of aformat of a fixed-length history stream shown in FIGS. 40 to 46 or avariable-length history stream shown in FIG. 47 to be described later.If the fixed-length history stream is selected on the encoder side,there is a merit that a circuit and software employed in the decoder fordecoding data elements from the history stream become simple. If thevariable-length history stream is selected on the encoder side, on theother hand, the encoder is capable of selecting arbitrarily historyinformation or data elements described in the user area of the picturelayer when necessary. Thus, the amount of data of the history stream canbe reduced. As a result, the data rate of the bitstream as a whole canalso be lowered as well.

The history information, the history data and the history parametercited in the explanation of the present invention are encodingparameters or data elements used in the related art encoding processesand are not encoding-parameter data used in the current encoding processor the encoding process carried out at the last stage. Consider a casein which a picture is encoded and transmitted as an I-picture in anencoding process of a first generation, as a P-picture in an encodingprocess of a second generation and as a B-picture in an encoding processof a third generation. Encoding parameters used in the encoding processof the third generation are described at predetermined locations of thesequence, GOP-picture, slice and macroblock layers of an encodedbitstream generated as a result of the encoding process of the thirdgeneration. On the other hand, encoding parameters used in the encodingprocess of the first and second generations are not recorded in thesequence or GOP layer for recording the encoding parameters used in theencoding process of the third generation, but recorded in the user-dataarea of the picture layer as history information of the encodingparameters.

First of all, the syntax of the fixed-length history stream is explainedby referring to FIGS. 40 to 46.

In the first place, encoding parameters related to the sequence headerof the sequence layer used in the previous encoding processes, that is,the encoding processes of typically the first and second generations,are inserted as a history stream into the user-data area of the picturelayer of the bitstream generated in the encoding process carried out atthe last stage, that is, the encoding process of typically the thirdgeneration. It should be noted that history information related to thesequence header of the sequence layer of the bitstream generated in theprevious encoding processes is never inserted into the sequence headerof the sequence layer of the bitstream generated in the encoding processcarried out at the last stage.

Data elements related to the sequence header used in the previousencoding processes include sequence_header_code,sequence_header_present_flag, horizontal_size_value,vertical_size_value, aspect_ratio_information, frame_rate_code,bit_rate_value, marker_bit, VBV_buffer_size_value,constrained_parameter_flag, load_intra_quantizer_matrix,intra_quantizer_matrix, load_non_intra_quantizer_matrix andnon_intra_quantizer_matrix as shown in FIG. 40.

The data elements listed above are described as follows. Thesequence_header_code data element is a start synchronization code of thesequence layer. The sequence_header_present_flag data element is a flagused for indicating whether data in the sequence header is valid orinvalid. The horizontal_size_value data element is data comprising thelow-order 12 bits of the number of pixels of the picture in thehorizontal direction. The vertical_size_value data element is datacomprising low-order 12 bits of the number of pixels of the picture inthe vertical direction. The aspect_ratio_information data element is anaspect ratio, that is, a ratio of the height to the width of thepicture, or the aspect ratio of the display screen. The frame_rate_codedata element is data representing the picture display period.

The bit_rate_value data element is data comprising the low-order 18 bitsof a bit rate for limiting the number of generated bits. The data isrounded up in 400-bsp units. The marker_bit data element is bit datainserted for preventing start-code emulation. The VBV_buffer_size_valuedata element is data comprising the low-order 10 bits of a value fordetermining the size of a virtual buffer (video buffer verifier) used incontrol of the amount of generated code. The constrained_parameter_flagdata element is a flag used for indicating whether or not parameters areunder constraint. The load_intra_quantizer_matrix data element is a flagused for indicating whether or not data of an intra-MB quantizationmatrix exists. The intra_quantizer_matrix data element is the value ofthe intra-MB quantization matrix. The load_non_intra_quantizer_matrixdata element is a flag used for indicating whether or not data of anon-intra-MB quantization matrix exists. The non_intra_quantizer_matrixdata element is the value of the non-intra-MB quantization matrix.

Subsequently, data elements representing a sequence extension used inthe previous encoding processes are described as a history stream in theuser area of the picture layer of the bitstream generated in theencoding process carried out at the last stage.

Data elements representing the sequence extension used in the previousencoding include extension_start_code, extension_start_code_identifier,sequence_extension_present_flag, profile_and_level_identification,progressive_sequence, chroma_format, horizontal_size_extension,vertical_size_extension, bit_rate_extension, vbv_buffer_size_extension,low_delay, frame_rate_extension_n and frame_rate_extension_d as shown inFIGS. 40 and 41.

The data elements listed above are described as follows. Theextension_start_code data element is a start synchronization code ofextension data. The extension_start_code_identifier data element is dataused for indicating which extension data is transmitted. Thesequence_extension_present_flag data element is a flag used forindicating whether data in the sequence extension is valid or invalid.The profile_and_level_identification data element is data specifying aprofile and a level of the video data. The progressive_sequence dataelement is data showing that the video data has been obtained fromsequential scanning. The chroma_format data element is data specifyingthe color-difference format of the video data.

The horizontal_size_extension data element is the two high-order bitsdata to be added to horizontal_size_value of the sequence header. Thevertical_size_extension data element is the two high-order bits data tobe added to vertical_size_value of the sequence header. Thebit_rate_extension data element is the twelve high-order bits data to beadded to bit_rate_value of the sequence header. Thevbv_buffer_size_extension data element is the eight high-order bits datato be added to vbv_buffer size_value of the sequence header. Thelow_delay data element is data used for indicating that a B-picture isnot included. The frame_rate_extension_n data element is data used forobtaining a frame rate in conjunction with frame_rate_code of thesequence header. The frame_rate_extension_d data element is data usedfor obtaining a frame rate in conjunction with frame_rate_code of thesequence header.

Subsequently, data elements representing a sequence-display extension ofthe sequence layer used in the previous encoding processes are describedas a history stream in the user area of the picture layer of thebitstream.

Data elements described as a sequence-display extension areextension_start_code, extension_start_code_identifier,sequence_display_extension_present_flag, video_format, color_decription,color_primaries, transfer_characteristics, matrix_coefficients,display_horizontal_size and display_vertical_size as shown in FIG. 41.

The data elements listed above are described as follows. Theextension_start_code data element is a start synchronization code ofextension data. The extension_start_code_identifier data element is dataused for indicating which extension data is transmitted. Thesequence_display_extension_presentation_flag data element is a flag usedfor indicating whether data elements in the sequence extension are validor invalid. The video_format data element is data representing the videoformat of the source signal. The color_description data element is dataused for indicating that detailed data of a color space exists. Thecolor_primaries data element is data showing details of a colorcharacteristic of the source signal. The transfer_characteristics dataelement is data showing details of how opto-electrical conversion hasbeen carried out. The matrix_coefficients data element is data showingdetails of how a source signal has been converted from the three primarycolors of light. The display_horizontal_size data element is datarepresenting the activation area or the horizontal size of an intendeddisplay. The display_vertical_size data element is data representing theactivation area or the vertical size of the intended display.

Subsequently, macroblock assignment data (namedmacroblock_assignment_in_user_data) showing phase information of amacroblock generated in the previous encoding processes is described asa history stream in the user area of the picture layer of a bitstreamgenerated in the encoding process carried out at the last stage.

The macroblock_assignment_in_user_data showing phase information of amacroblock comprises data elements such asmacroblock_assignment_present_flag, v_phase and h_phase as shown in FIG.41.

The data elements listed above are described as follows. Themacroblock_assignment_present_flag data element is a flag used forindicating whether data elements of macroblock_assignment_in_user_dataare valid or invalid. The v_phase data element is data showing phaseinformation in the vertical direction which is obtained when themacroblock is detached from picture data. The h_phase data element isdata showing phase information in the horizontal direction which isobtained when the macroblock is detached from picture data.

Subsequently, data elements representing a GOP header of the GOP layerused in the previous encoding processes are described as a historystream in the user area of the picture layer of a bitstream generated inthe encoding process carried out at the last stage.

The data elements representing the GOP header are group_start_code,group_of_picture_header_present_flag, time_code, closed_gop andbroken_link as shown in FIG. 41.

The data elements listed above are described as follows. Thegroup_start_code data element is the start synchronization code of theGOP layer. The group_of_picture_header_present_flag data element is aflag used for indicating whether data elements ingroup_of_picture_header are valid or invalid. The time_code data elementis a time code showing the length of time measured from the beginning ofthe first picture of the GOP. The closed_gop data element is a flag usedfor indicating whether or not it is possible to carry out an independentplayback operation of a picture in one GOP from another GOP. Thebroken_link data element is a flag used for indicating whether or notthe B-picture at the beginning of the GOP can not be reproduced with ahigh degree of accuracy because of reasons such as editing.

Subsequently, data elements representing a picture header of the picturelayer used in the previous encoding processes are described as a historystream in the user area of the picture layer of a bitstream generated inthe encoding process carried out at the last stage.

The data elements related to a picture header are picture_start_code,temporal_reference, picture_coding_type, vbv_delay,full_pel_forward_vector, forward_f_code, full_pel_backward_vector andbackward_f_code as shown in FIGS. 41 and 42.

The data elements listed above are described concretely as follows. Thepicture_start_code data element is the start synchronization code of thepicture layer. The temporal_reference data element is a number used forindicating a display order of the picture. This number is reset at thebeginning of the GOP. The picture_coding_type data element is data usedfor indicating the type of the picture. The vbv_delay data element isdata showing an initial state of a virtual buffer at a random access.The full_pel_forward_vector data element is a flag used for indicatingwhether the precision of the forward motion vector is expressed in termsof pixel units or half-pixel units. The forward_f_code data element isdata representing a forward-motion-vector search range. Thefull_pel_backward_vector data element is a flag used for indicatingwhether the precision of the backward motion vector is expressed interms of pixel units or half-pixel units. The backward_f_code dataelement is data representing a backward-motion-vector search range.

Subsequently, data elements representing a picture-coding extension ofthe picture layer used in the previous encoding processes are describedas a history stream in the user area of the picture layer of a bitstreamgenerated in the encoding process carried out at the last stage.

The data elements related to the picture-coding extension areextension_start_code, extension_start_code_identifier, f_code[0] [0],f_code[0] [1], f_code[1][0], f_code[1][1], intra_dc_precision,picture_structure, top_field_first, frame_predictive_frame_dct,concealment_motion_vectors, q_scale_type, intra_vlc_format,alternate_scan, repeat_first_field, chroma_420_type, progressive_frame,composite_display_flag, v_axis, field_sequence, sub_carrier,burst_amplitude and sub_carrier_phase as shown in FIG. 42.

The data elements listed above are described as follows. Theextension_start_code data element is a start code used for indicatingthe start of extension data of the picture layer. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. The f_code[0] [0] dataelement is data representing a horizontal motion-vector search range inthe forward direction. The f_code[0] [1] data element is datarepresenting a vertical motion-vector search range in the forwarddirection. The f_code[1] [0] data element is data representing ahorizontal motion-vector search range in the backward direction. Thef_code[1] [1] data element is data representing a vertical motion-vectorsearch range in the backward direction.

The intra_dc_precision data element is data representing the precisionof DC coefficients. The picture_structure data element is data used forindicating whether the data structure is a frame structure or a fieldstructure. In the case of the field structure, the picture_structuredata element also indicates whether the field structure is thehigh-order field or the low-order field. The top_field_first dataelement is data used for indicating whether the first field of a framestructure is the high-order field or the low-order field. Theframe_predictive_frame_dct data element is data used for indicating thatthe prediction of frame-mode DCT is carried out only in the frame-DCTmode in the case of a frame structure. The concealment_motion_vectorsdata element is data used for indicating that the intra-macroblockincludes a motion vector for concealing a transmission error.

The q_scale_type data element is data used for indicating whether to usea linear quantization scale or a non-linear quantization scale. Theintra_vlc_format data element is data used for indicating whether or notanother 2-dimensional VLC is used in the intra-macroblock. Thealternate_scan data element is data representing selection to use azigzag scan or an alternate scan. The repeat_first_field data element isdata used in the case of a 2:3 pull-down. The chroma_420_type dataelement is data equal to the value of the next progressive_frame dataelement in the case of a 4:2:0 signal format or 0 otherwise. Theprogressive_frame data element is data used for indicating whether ornot this picture has been obtained from sequential scanning. Thecomposite_display_flag data element is a flag used for indicatingwhether or not the source signal is a composite signal.

The v_axis data element is data used in the case of a PAL source signal.The field_sequence data element is data used in the case of a PAL sourcesignal. The sub_carrier data element is data used in the case of a PALsource signal. The burst_amplitude data element is data used in the caseof a PAL source signal. The sub_carrier_phase data element is data usedin the case of a PAL source signal.

Subsequently, a quantization-matrix extension used in the previousencoding processes is described as a history stream in the user area ofthe picture layer of the bitstream generated in the encoding processcarried out at the last stage.

Data elements related to the quantization-matrix extension areextension_start_code, extension_start_code_identifier,quant_matrix_extension_present_flag, load_intra_quantizer_matrix,intra_quantizer_matrix[64], load_non_intra_quantizer_matrix,non_intra_quantizer_matrix[64], load_chroma_intra_quantizer_matrix,chroma_non_intra_quantizer_matrix[64],load_chroma_intra_quantizer_matrix andchroma_non_intra_quantizer_matrix[64] as shown in FIG. 43.

The data elements listed above are described as follows. Theextension_start_code data element is a start code used for indicatingthe start of the quantization-matrix extension. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. Thequant_matrix_extension_present_flag data element is a flag used forindicating whether data elements of the quantization-matrix extensionare valid or invalid. The load_intra_quantizer_matrix data element isdata used for indicating whether or not quantization-matrix data for anintra-macroblock exists. The intra_quantizer_matrix data element is datarepresenting values of a quantization-matrix for an intra-macroblock.

The load_non_intra_quantizer_matrix data element is data used forindicating whether or not quantization-matrix data for anon-intra-macroblock exists. The non_intra_quantizer_matrix data elementis data representing values of a quantization-matrix for anon-intra-macroblock. The load_chroma_intra_quantizer_matrix dataelement is data used for indicating whether or not quantization-matrixdata for a color-difference intra-macroblock exists. Thechroma_intra_quantizer_matrix data element is data representing valuesof a quantization-matrix for a color-difference intra-macroblock. Theload_chroma_non_intra_quantizer_matrix data element is data used forindicating whether or not quantization-matrix data for acolor-difference non-intra-macroblock exists. Thechroma_non_intra_quantizer_matrix data element is data representingvalues of a quantization-matrix for a color-differencenon-intra-macroblock.

Subsequently, a copyright extension used in the previous encodingprocesses is described as a history stream in the user area of thepicture layer of the bitstream generated in the encoding process carriedout at the last stage.

Data elements related to the copyright extension areextension_start_code, extension_start_code_identifier,copyright_extension_present_flag, copyright_flag, copyright_identifier,original_or_copy, copyright_number_1, copy_right number_2 andcopyright_number_3 as shown in FIG. 43.

The data elements listed above are described as follows. Theextension_start_code data element is a start code used for indicatingthe start of the copyright extension. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. Thecopyright_extension_present_flag data element is a flag used forindicating whether data elements of the copyright extension are valid orinvalid. The copyright_flag data element is a flag used for indicatingwhether or not a copyright has been given to encoded video data in arange up to the next copyright extension or the end of the sequence.

The copyright_identifier data element is data used for identifying aninstitution cataloging the copyright specified by the ISO/IEC JTC/SC29.The original_or_copy data element is a flag used for indicating whetherdata of the bitstream is original or copied data. The copyright_number_1data element indicates bits 44 to 63 of a copyright number. Thecopyright_number_2 data element indicates bits 22 to 43 of the copyrightnumber. The copyright_number_3 data element indicates bits 0 to 21 ofthe copyright number.

Subsequently, a picture-display extension (picture_display_extension)used in the previous encoding processes is described as a history streamin the user area of the picture layer of the bitstream generated in theencoding process carried out at the last stage.

Data elements representing the picture-display extension areextension_start_code, extension_start_code_identifier,picture_display_extension_present_flag,frame_center_horizontal_offset_1, frame_center_vertical_offset_1,frame_center_horizontal_offset_2, frame_center_vertical_offset_2,frame_center_horizontal_offset_3 and frame_center_vertical_offset_3 asshown in FIG. 44.

The data elements listed above are described as follows. Theextension_start_code data element is a start code used for indicatingthe start of the picture-display extension. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. Thepicture_display_extension_present_flag data element is a flag used forindicating whether data elements of the picture-display extension arevalid or invalid. The frame_center_horizontal_offset data element is anoffset of the display area in the horizontal direction and theframe_center_vertical_offset data element is an offset of the displayarea in the vertical direction. Up to three offset value of horizontaland vertical offsets can be defined respectively.

User data is described as a history stream after the history informationrepresenting the picture-display extension already explained in the userarea of the picture layer of the bitstream generated in the encodingprocess carried out at the last stage as shown in FIG. 44.

Following to the user data, information on a macroblock used in theprevious encoding processes is described as a history stream as shown inFIGS. 44 to 46.

Information on the macroblock comprises data elements related to theposition of the macroblock, data elements related to the mode of themacroblock, data elements related to control of the quantization step,data elements related to motion compensation, data elements related tothe pattern of the macroblock and data elements related to the amount ofgenerated code as shown in FIGS. 44 to 46. The data elements related tothe position of the macroblock include such as macroblock_address_h,macroblock_address_v, slice_header_present_flag andskipped_macroblock_flag. The data elements related to the mode of themacroblock include such as macroblock_quant, macroblock_motion_forward,macroblock_motion_backward, macroblock_pattern, macroblock_intra,spatial_temporal_weight_code_flag, frame_motion_type and dct_type. Thedata elements related to control of the quantization step includes suchas quantiser_scale_code. The data elements related to motioncompensation include PMV[0] [0] [0], PMV[0] [0] [1],motion_vertical_field_select[0] [0], PMV[0] [1] [0], PMV[0] [1] [1],motion_vertical_field_select[0] [1], PMV[1] [1] [0], PMV[1] [0] [1],motion_vertical_field_select[1] [0], PMV[1] [1] [0], PMV[1] [1] [1] andmotion_vertical_field_select [1] [1]. The data elements related to thepattern of the macroblock include such as coded_block_pattern, and thedata elements related to the amount of generated code are num_mv_bits,num_coef_bits and num_other_bits or the like.

The data elements-related to the macroblock are described in detail asfollows.

The macroblock_address_h data element is data defining the presentabsolute position of the macroblock in horizontal direction. Themacroblock_address_v data element is data defining the present absoluteposition of the macroblock in vertical direction. Theslice_header_present_flag data element is a flag used for indicatingthis macroblock is located at the beginning of a slice layer, andwhether or not being accompanied by a slice header. Theskipped_macroblock_flag data element is a flag used for indicatingwhether or not to skip this macroblock in a decoding process.

The macroblock_quant data element is data derived from macroblock_typeshown in FIGS. 65 to 67. This data element indicates whether or notquantiser_scale_code appears in the bitstream. Themacroblock_motion_forward data element is data derived from themacroblock_type shown in FIGS. 65 to 67 and used in the decodingprocess. The macroblock_motion_backward data element is data derivedfrom the macroblock_type shown in FIGS. 65 to 67 and used in thedecoding process. The macroblock_pattern data element is data derivedfrom the macroblock_type shown in FIGS. 65 to 67 and it indicateswhether or not coded_block_pattern appears in the bitstream.

The macroblock_intra data element is data derived from themacroblock_type shown in FIGS. 65 to 67 and used in the decodingprocess. The spatial_temporal_weight_code_flag data element is a flagderived from the macrobloc_type shown in FIGS. 65 to 67 and used forindicating whether or not spatial_temporal_weight_code showing anup-sampling technique of a low-order layer picture with time scalabilityexists in the bitstream.

The frame_motion_type data element is a 2-bit code used for indicatingthe prediction type of the macroblock of a frame. A frame_motion_typevalue of “00” indicates that there are two prediction vectors and theprediction type is a field-based prediction type. A frame_motion_typevalue of “01” indicates that there is one prediction vector and theprediction type is a field-based prediction type. A frame_motion_typevalue of “10” indicates that there is one prediction vector and theprediction type is a frame-based prediction type. A frame_motion_typevalue of “11” indicates that there is one prediction vector and theprediction type is a dual-prime prediction type. The field_motion_typedata element is a 2-bit code showing the motion prediction of themacroblock of a field. A field_motion_type value of “01” indicates thatthere is one prediction vector and the prediction type is a field-basedprediction type. A field_motion_type value of “10” indicates that thereis two prediction vectors and the prediction type is a 18×8macroblock-based prediction type. A field_motion_type value of “11”indicates that there is one prediction vector and the prediction type isa dual-prime prediction type. The dct_type data element is data used forindicating whether the DCT is carried out in the frame-DCT mode or thefield-DCT mode. The quantiser_scale_code data element indicates thequantization-step size of the macroblock.

Next, data elements related to a motion vector are described. In orderto reduce the magnitude of a motion vector required in a decodingprocess, a particular motion vector is subjected to an encoding processby actually encoding a difference between the particular motion vectorand a motion vector decoded earlier. A decoder for decoding a motionvector has to sustain four motion-vector prediction values, each ofwhich comprises horizontal and vertical components. These motion-vectorprediction values are represented by PMV[r] [s] [v]. The subscript [r]is a flag used for indicating whether the motion vector in a macroblockis the first or second vector. To be more specific, an [r] value of “0”indicates the first vector and an [r] value of “1” indicates the secondvector. The subscript [s] is a flag used for indicating whether thedirection of the motion vector in the macroblock is the forward orbackward direction. To be more specific, an [s] value of “0” indicatesthe forward direction of the motion vector and an [r] value of “1”indicates the backward direction of the motion vector. The subscript [v]is a flag used for indicating whether the component of the motion vectorin the macroblock is a component in the horizontal or verticaldirection. To be more specific, a [v] value of “0” indicates thehorizontal component of the motion vector and a [v] value of “1”indicates the vertical component of the motion vector.

Thus, PMV[0] [0] [0] is data representing the horizontal component, theforward motion vector of the first vector. PMV[0] [0] [1] is datarepresenting the vertical component of the forward motion vector of thefirst vector. PMV[0] [1] [0] is data representing the horizontalcomponent of the backward motion vector of the first vector. PMV[0] [1][1] is data representing the vertical component of the backward motionvector of the first vector. PMV[1] [0] [0] is data representing thehorizontal component of the forward motion vector of the second vector.PMV[1] [0] [1] is data representing the vertical component of theforward motion vector of the second vector. PMV[1] [1] [0] is datarepresenting the horizontal component of the backward motion vector ofthe second vector. PMV[1] [1] [1] is data representing the verticalcomponent of the backward motion vector of the second vector.

A motion_vertical_field_select[r] [s] is data used for indicating whichreferenced field of the prediction format is used. To be more specific,a motion_vertical_field_select[r] [s]-value of “0” indicates the topreferenced field and a motion_vertical_field_select[r] [s]-value of “1”indicates the bottom referenced field to be used.

In motion_vertical_field_select[r] [s], the subscript [r] is a flag usedfor indicating whether the motion vector in a macroblock is the first orsecond vector. To be more specific, an [r] value of “0” indicates thefirst vector and an [r] value of “1” indicates the second vector. Thesubscript [s] is a flag used for indicating whether the direction of themotion vector in the macroblock is the forward or backward direction. Tobe more specific, an [s] value of “0” indicates the forward direction ofthe motion vector and an [r] value of “1” indicates the backwarddirection of the motion vector. Thus, motion_vertical_field_select[0][0] indicates the referenced field used in the generation of the forwardmotion vector of the first vector. motion_vertical_field_select[0] [1]indicates the referenced field used in the generation of the backwardmotion vector of the first vector. motion_vertical_field_select[1] [0]indicates the referenced field used in the generation of the forwardmotion vector of the second vector. motion_vertical_field_select[1] [1]indicates the referenced field used in the generation of the backwardmotion vector of the second vector.

The coded_block_pattern data element is variable-length data used forindicating which DCT block among a plurality of DCT blocks each forstoring a DCT coefficient contains a meaningful or non-zero DCTcoefficient. The num_mv_bits data element is data representing theamount of code of the motion vector in the macroblock. The num_coef_bitsdata element is data representing the amount of code of the DCTcoefficient in the macroblock. The num_other_bits data element shown inFIG. 46 is data representing the amount of code in the macroblock otherthan the motion vector and the DCT coefficient.

Next, a syntax for decoding data elements from a history stream with avariable length is explained by referring to FIGS. 47 to 64.

As shown in FIG. 47, the history stream with a variable length comprisesdata elements defined by a next_start_code( ) function, asequence_header( ) function, a sequence_extension( ) function, anextension_and_user_data(0) function, a group_of_picture_header( )function, an extension_and_user_data(1) function, a picture_header( )function, a picture_coding_extension( ) function, anextension_and_user_data(2) function and a picture_data( ) function.

Since the next_start_code( ) function is a function used for searching abitstream for a start code, data elements defined by thesequence_header( ) function and used in the previous encoding processesare described at the beginning of the history stream as shown in FIG.48.

Data elements defined by the sequence_header( ) function includesequence_header_code, sequence_header_present_flag,horizontal_size_value, vertical_size_value, aspect_ratio_information,frame_rate_code, bit_rate_value, marker_bit, VBV_buffer_size_value,constrained_parameter_flag, load_intra_quantizer_matrix,intra_quantizer_matrix, load_non_intra_quantizer_matrix andnon_intra_quantizer_matrix as shown in FIG. 48.

The data elements listed above are described as follows. Thesequence_header_code data element is the start synchronization code ofthe sequence layer. The sequence_header_present_flag data element is aflag used for indicating whether data in sequence_header is valid orinvalid. The horizontal_size_value data element is data comprising thelow-order 12 bits of the number of pixels of the picture in thehorizontal direction. The vertical_size_value data element is datacomprising the low-order 12 bits of the number of pixels of the picturein the vertical direction. The aspect_ratio_information data element isan aspect ratio of pixels of a picture, that is, a ratio of the heightto the width of the picture, or the aspect ratio of the display screen.The frame_rate_code data element is data representing the picturedisplay period. The bit_rate_value data element is data comprising thelow-order 18 bits of a bit rate for limiting the number of generatedbits. The data is rounded up in 400-bsp units.

The marker_bit data element is bit data inserted for preventingstart-codes emulation. The VBV_buffer_size_value data element is datacomprising the low-order 10 bits of a value for determining the size avirtual buffer (video buffer verifier) used in control of the amount ofgenerated code. The constrained_parameter_flag data element is a flagused for indicating whether or not parameters are under constraint. Theload_intra_quantizer_matrix data element is a flag used for indicatingwhether or not data of an intra-MB quantization matrix exists. Theintra_quantizer_matrix data element is the value of the intra-MBquantization matrix. The load_non_intra_quantizer_matrix data element isa flag used for indicating whether or not data of a non-intra-MBquantization matrix exists. The non_intra_quantizer_matrix data, elementis the value of the non-intra-MB quantization matrix.

Following to the data elements defined by the sequence_header( )function, data elements defined by the sequence_extension( ) functionare described as a history stream as shown in FIG. 49.

The data elements defined by the sequence_extension( ) function includeextension_start_code, extension_start_code_identifier,sequence_extension_present_flag, profile_and_level_identification,progressive_sequence, chroma_format, horizontal_size_extension,vertical_size_extension, bit_rate_extension, vbv_buffer_size_extension,low_delay, frame_rate_extension_n and frame_rate_extension_d as shown inFIG. 49.

The data elements listed above are described as follows. Theextension_start_code data element is a start synchronization code ofextension data. The extension_start_code_identifier data element is dataused for indicating which extension data is transmitted. Thesequence_extension_present_flag data element is a flag used forindicating whether data in the sequence extension is valid or invalid.The profile_and_level_identification data element is data specifying aprofile and a level of the video data. The progressive_sequence dataelement is data showing that the video data has been obtained fromsequential scanning. The chroma_format data element is data specifyingthe color-difference format of the video data. Thehorizontal_size_extension data element is data to be added tohorizontal_size_value of the sequence header as the two high-order bits.The vertical_size_extension data element is data to be added tovertical_size_value of the sequence header as the two high-order bits.The bit_rate_extension data element is data to be added tobit_rate_value of the sequence header as the 12 high-order bits. Thevbv_buffer_size_extension data element is data to be added tovbv_buffer_size_value of the sequence header as the 8 high-order bits.

The low_delay data element is data used for indicating that a B-pictureis not included. The frame_rate_extension_n data element is data usedfor obtaining a frame rate in conjunction with frame_rate_code of thesequence header. The frame_rate_extension_d data element is data usedfor obtaining a frame rate in conjunction with frame_rate_code of thesequence header.

Following to the data elements defined by the sequence_extension( )function, data elements defined by, the extension_and_user_data(0)function are described as a history stream as shown in FIG. 50. For (i)with a value other than 2, the extension_and_user_data(i) functiondescribes only data elements defined by a user_data( ) function as ahistory stream instead of describing data elements defined by theextension_data( ) function. Thus, the extension_and_user_data(0)function describes only data elements defined by the user_data( )function as a history stream.

The user_data( ) function describes user data as a history stream on thebasis of a syntax like one shown in FIG. 51.

Following to the data elements defined by the extension_and_user_data(0)function, data elements defined by the group_of_picture_header( )function shown in FIG. 52 and data elements defined by theextension_and_user_data(1) function shown in FIG. 50 are described as ahistory stream. It should be noted, however, that the data elementsdefined by the group_of_picture_header( ) function and data elementsdefined by the extension_and_user_data(1) function are described only ifgroup_start_code representing the start code of the GOP layer isdescribed in the history stream.

As shown in FIG. 52, the data elements defined by thegroup_of_picture_header( ) function are group_start_code,group_of_picture_header_present_flag, time_code, closed_gop andbroken_link.

The data elements listed above are described as follows. Thegroup_start_code data element is the start synchronization code of theGOP layer. The group_of_picture_header_present_flag data element is aflag used for indicating whether data elements ingroup_of_picture_header are valid or invalid. The time_code data elementis a time code showing the length of time measured from the beginning ofthe first picture of the GOP. The closed_gop data element is a flag usedfor indicating whether or not it is possible to carry out an independentplayback operation of a picture in the GOP from another GOP. Thebroken_link data element is a flag used for indicating that theB-picture at the beginning of the GOP can not be reproduced with a highdegree of accuracy because of reasons such as editing.

Much like the extension_and_user_data(0) function shown in FIG. 50, theextension_and_user_data(1) function describes only data elements definedby the user_data( ) function as a history stream.

If group_start_code representing the start code of the GOP layer is notdescribed in the history stream, the data elements defined by thegroup_of_picture_header( ) function and data elements defined by theextension_and_user_data(1) function are also not described in thehistory stream. In this case, data elements defined by thepicture_header( ) function are described after the data elements definedby the extension_and_user_data(0) function.

The data elements defined by the picture_headr( ) function arepicture_start_code, temporal_reference, picture_coding_type, vbv_delay,full_pel_forward_vector, forward_f_code, full_pel_backward_vector,backward_f_code, extra_bit_picture and extra_information_picture asshown in FIG. 53.

The data elements listed above are described concretely as follows. Thepicture_start_code data element is the start synchronization code of thepicture layer. The temporal_reference data element is a number used forindicating a display order of the picture. This number is reset at thebeginning of the GOP. The picture_coding_type data element is data usedfor indicating the type of the picture. The vbv_delay data element isdata showing an initial state of a virtual buffer at a random access.The full_pel_forward_vector data element is a flag used for indicatingwhether the precision of the forward motion vector is expressed in termsof integral pixel units or half-pixel units. The forward_f_code dataelement is data representing a forward-motion-vector search range. Thefull_pel_backward_vector data element is a flag used for indicatingwhether the precision of the backward motion vector is expressed interms of integral pixel units or half-pixel units. The backward_f_codedata element is data representing a backward-motion-vector search range.The extra_bit_picture data element is a flag used for indicating whetheror not following additional information exists. To be more specific,extra_bit_picture having a value of “0” indicates that no followingadditional information exists while extra_bit_picture having a value of“1” indicates that following additional information exists. Theextra_information_picture data element is information reserved byspecifications.

Following to the data elements defined by the picture_headr( ) function,data elements defined by the picture_coding_extension( ) function shownin FIG. 54 are described as a history stream.

The data elements defined by the picture_coding_extension( ) functionare extension_start_code, extension_start_code_identifier, f_code[0][0], f_code[0] [1], f_code[1] [0], f_code[1] [1], intra_dc_precision,picture_structure, top_field_first, frame_predictive_frame_dct,concealment_motion_vectors, q_scale_type, intra_vlc_format,alternate_scan, repeat_first_field, chroma_420_type, progressive_frame,composite_display_flag, v_axis, field_sequence, sub_carrier,burst_amplitude and sub_carrier_phase as shown in FIG. 54.

The data elements listed above are described as follows. Theextension_start_code data element is a start code used for indicatingthe start of extension data of the picture layer. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. The f_code[0] [0] dataelement is data representing a horizontal motion-vector search range inthe forward direction. The f_code[0] [1] data element is datarepresenting a vertical motion-vector search range in the forwarddirection. The f_code[1] [0] data element is data representing ahorizontal motion-vector search range in the backward direction. Thef_code[1] [1] data element is data representing a vertical motion-vectorsearch range in the backward direction. The intra_dc_precision dataelement is data representing the precision of DC coefficients.

The picture_structure data element is data used for indicating whetherthe data structure is a frame structure or a field structure. In thecase of the field structure, the picture_structure data element alsoindicates whether the field structure is the high-order field or thelow-order field. The top_field_first data element is data used forindicating whether the first field of a frame structure is thehigh-order field or the low-order field. The frame_predictive_frame_dctdata element is data used for indicating that the prediction offrame-mode DCT is carried out only in the frame mode in the case of aframe structure. The concealment_motion_vectors data element is dataused for indicating that the intra-macroblock includes a motion vectorfor concealing a transmission error. The q_scale_type data element isdata used for indicating whether to use a linear quantization scale or anon-linear quantization scale. The intra_vlc_format data element is dataused for indicating whether or not another 2-dimensional VLC is used inthe intra-macroblock.

The alternate_scan data element is data representing selection to use azigzag scan or an alternate scan. The repeat_first_field data element isdata used in the case of a 2:3 pull-down. The chroma_420_type dataelement is data equal to the value of the next progressive_frame dataelement in the case of a 4:2:0 signal format or 0 otherwise. Theprogressive_frame data element is data used for indicating whether ornot this picture has been obtained from sequential scanning. Thecomposite_display_flag data element is data used for indicating whetheror not the source signal is a composite signal. The v_axis data elementis data used in the case of a PAL source signal. The field_sequence dataelement is data used in the case of a PAL source signal. The sub_carrierdata element is data used in the case of a PAL source signal. Theburst_amplitude data element is data used in the case of a PAL sourcesignal. The sub_carrier_phase data element is data used in the case of aPAL source signal.

Following to the data elements defined by the picture_coding_extension() function, data elements defined by the extension_and user_data(2)function shown in FIG. 50 are described as a history stream. It shouldbe noted, however, that data elements defined by the extension_data( )function are described by the extension_and_user_data(2) function onlyif extension_start_code representing the start code of the extensionexists in the bit stream. In addition, data elements defined by theuser_data( ) function are described by the extension_and_user_data(2)function after the data elements defined by the extension_data( )function only if user_data_start_code representing the start code of theuser data exists in the bitstream as shown in as shown in FIG. 50. Thatis to say, if neither the start code of the extension nor the start codeof the user data exists in the bitstream, data elements defined by theextension_data( ) function and data elements defined by the user_data( )function are not described in the bitstream.

The extension_data( ) function is a function used for describing a dataelement representing extension_start_code and data elements defined by aquant_matrix_extension( ) function, a copyright_extension( ) functionand a picture_display_extension( ) function as a history stream in thebitstream as shown in FIG. 55.

Data elements defined by the quant_matrix_extension( ) function areextension_start_code, extension_start_code_identifier,quant_matrix_extension_present_flag, load_intra_quantizer_matrix,intra_quantizer_matrix[64], load_non_intra_quantizer_matrix,non_intra_quantizer_matrix[64], load_chroma_intra_quantizer_matrix,chroma_intra_quantizer_matrix[64],load_chroma_non_intra_quantizer_matrix andchroma_non_intra_quantizer_matrix[64] as shown in FIG. 56.

The data elements listed above are described as follows. Theextension_start_code data element is a start code used for indicatingthe start of the quantization-matrix extension. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. Thequant_matrix_extension_present_flag data element is a flag used forindicating whether data elements of the quantization-matrix extensionare valid or invalid. The load_intra_quantizer_matrix data element isdata used for indicating whether or not quantization-matrix data for anintra-macroblock exists. The intra_quantizer_matrix data element is datarepresenting values of a quantization-matrix for an intra-macroblock.

The load_non_intra_quantizer_matrix data element is data used forindicating whether or not quantization-matrix data for anon-intra-macroblock exists. The non_intra_quantizer_matrix data elementis data representing values of a quantization-matrix for anon-intra-macroblock. The load_chroma_intra_quantizer_matrix dataelement is data used for indicating whether or not quantization-matrixdata for a color-difference intra-macroblock exists. Thechroma_intra_quantizer_matrix data element is data representing valuesof a quantization-matrix for a color-difference intra-macroblock. Theload_chroma_non_intra_quantizer_matrix data element is data used forindicating whether or not quantization-matrix data for acolor-difference non-intra-macroblock exists. Thechroma_non_intra_quantizer_matrix data element is data representingvalues of a quantization-matrix for a color-differencenon-intra-macroblock.

The data elements defined by the copyright_extension( ) function areextension_start_code, extension_start_code_identifier,copyright_extension_present_flag, copyright_flag, copyright_identifier,original_or_copy, copyright_number_1, copy_right_number_2 andcopyright_number_3 as shown in FIG. 57.

The data elements listed above are described as follows. Theextension_start_code data element is a start code used for indicatingthe start of the copyright extension. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. Thecopyright_extension_present_flag data element is a flag used forindicating whether data elements of the copyright extension are valid orinvalid.

The copyright_flag data element is a flag used for indicating whether ornot a copyright has been given to encoded video data in a range up tothe next copyright extension or the end of the sequence. Thecopyright_identifier data element is data used for identifying aninstitution cataloging the copyright specified by the ISO/IEC JTC/SC29.The original_or_copy data element is a flag used for indicating whetherdata of the bitstream is originator copied data. The copyright_number_1data element indicates bits 44 to 63 of a copyright number. Thecopyright_number_2 data element indicates bits 22 to 43 of the copyrightnumber. The copyright_number_3 data element indicates bits 0 to 21 ofthe copyright number.

The data elements defined by the picture_display_extension( ) functionare extension_start_code_identifier, frame_center_horizontal_offset andframe_center_vertical_offset as shown in FIG. 58.

The data elements listed above are described as follows. Theextension_start_code_identifier data element is a code used forindicating which extension data is transmitted. Theframe_center_horizontal_offset data element is an offset of the displayarea in the horizontal direction. The number of such horizontal offsetscan be defined by number_of_frame_center_offsets. Theframe_center_vertical_offset data element is an offset of the displayarea in the vertical direction. The number of such vertical offsets canbe defined by number_of_frame_center_offsets.

As shown in the variable-length history stream of FIG. 47, data elementsdefined by a picture_data( ) function are described as a history streamafter the data elements defined by the extension_and_user(2) function.

As shown in FIG. 59, the data elements defined by a picture_data( )function are data elements defined by a slice( ) function. It should benoted that the data elements defined by a slice( ) function are notdescribed in the bitstream if slice_start_code representing the startcode of the slice( ) function does not exist in the bitstream.

As shown in FIG. 60, the slice( ) function is a function used fordescribing data elements such as slice_start_code,slice_quantiser_scale_code, intra_slice_flag, intra_slice,reserved_bits, extra_bit_slice, extra_information_slice andextra_bit_slice and data elements defined by a macroblock( ) function asa history stream.

The data elements listed above are described as follows. Theslice_start_code data element is a start code used for indicating thedata elements defined by the slice( ) function. Theslice_quantiser_scale_code data element is the size of the quantizationstep defined for a macroblock existing in the slice layer. However,quantiser_scale_code set for macroblocks is preferably used, whenquantiser_scale_code has been set.

The intra_slice_flag data segment is a flag used for indicating whetheror not intra_slice and reserved_bits exist in the bit stream. Theintra_slice data element is a flag used for indicating whether or not anon-intra-macroblock exists in the slice layer. To be more specific, ifany one of macroblocks in the slice layer is a non-intra-macroblock, theintra_slice flag has a value of “0”. If all macroblocks in the slicelayer are non-intra-macroblocks, on the other hand, the intra_slice flaghas a value of “1”. The reserved_bits data element is 7-bit data havinga value of “0”. The extra_bit_slice data element is a flag used forindicating whether or not the extra_information_slice data element, thatis, information added as a history stream, exists. To be more specific,if the next extra_information_slice data element exists, theextra_bit_slice flag has a value of “1”. If the nextextra_information_slice data element does not exist, on the other hand,the extra_bit_slice flag has a value of “0”.

Following to the data element defined by the slice( ) function, dataelements defined by a macroblock( ) function are described as a historystream.

As shown in FIG. 61, the macroblock( ) function are a function used fordefining data elements such as macroblock_escape,macroblock_address_increment and macroblock_quantiser_scale_code anddata elements defined by a macroblock_modes( ) function and amacroblock_vectors(s) function.

The data elements listed above are described as follows. Themacroblock_escape data element is a string of bits with a fixed lengthused for indicating whether or not a difference in the horizontaldirection between a referenced macroblock and a preceding macroblock isat least 34 or greater. If the difference in the horizontal directionbetween a referenced macroblock and a preceding macroblock is at least34 or greater, 33 is added to the value of themacroblock_address_increment data element. Themacroblock_address_increment data element is the difference in thehorizontal direction between a referenced macroblock and a precedingmacroblock. If one macroblock_escape data element exists before themacroblock_address_increment data element, a value obtained as a resultof the addition of 33 to the value of the macroblock_address_incrementdata element represents the actual difference in the horizontaldirection between a referenced macroblock and a preceding macroblock.

The macroblock_quantiser_scale_code data element is the size of thequantization step set in each macroblock. The slice_quantiser_scale_codedata element representing the size of the quantization step of a slicelayer is also set in each slice layer. However, macroblock_scale_codeset for a macroblock takes precedence of slice_quantiser_scale_code.

Following to the macroblock_address_increment data element, dataelements defined by the macroblock_modes( ) function are described. Asshown in FIG. 62, the macroblock_modes( ) function is a function usedfor describing data elements such as macroblock_type, frame_motion_type,field_motion_type and dct_type as a history stream.

The data elements listed above are described as follows. Themacroblock_type data element is data representing the encoding type ofthe macroblock. To put it concretely, the macroblock_type data elementis data with a variable length generated from flags such asmacroblock_quant, dct_type_flag, macroblock_motion_forward andmacroblock_motion_backward as shown in FIGS. 65 to 67. Themacroblock_quant flag is a flag used for indicating whether or notmacroblock_quantiser_scale_code for setting the size of the quantizationstep for the macroblock is set. If macroblock_quantiser_scale_codeexists in the bitstream, the macroblock_quant flag has a value of “1”.

The dct_type_flag is a flag used for indicating whether or not dct_typeshowing that the referenced macroblock has been encoded in the frame-DCTmode or the field-DCT mode exists. In other words, dct_type_flag is aflag used for indicating whether or not the referenced macroblockexperienced DCT. If dct_type exists in the bitstream, dct_type_flag hasa value of “1”. The macroblock_motion_forward is a flag showing whetheror not the referenced macroblock has undergone forward prediction. Ifthe referenced macroblock has undergone forward prediction, themacroblock_motion_forward flag has a value of “1”. On the other hand,macroblock_motion_backward is a flag showing whether or not thereferenced macroblock has undergone backward prediction. If thereferenced macroblock has undergone backward prediction, themacroblock_motion_backward flag has a value of “1”.

If the macroblock_motion_forward flag or the macroblock_motion_backwardflag has a value of “1”, the picture is transferred in theframe-prediction mode and frame_period_frame_dct has a value of “0”, adata element representing frame_motion_type is described after a dataelement representing macroblock_type. It should be noted thatframe_period_frame_dct is a flag used for indicating whether or notframe_motion_type exists in the bit stream.

The frame_motion_type data element is a 2-bit code showing theprediction type of the macroblock of the frame. A frame_motion_typevalue of “00” indicates that there are two prediction vectors and theprediction type is a field-based prediction type. A frame_motion_typevalue of “01” indicates that there is one prediction vector and theprediction type is a field-based prediction type. A frame_motion_typevalue of “10” indicates that there is one prediction vector and theprediction type is a frame-based prediction type. A frame_motion_typevalue of “11” indicates that there is one prediction vector and theprediction type is a dual-prime prediction type.

If the macroblock_motion_forward flag or the macroblock_motion_backwardflag has a value of “1” and the picture is transferred not in the frameprediction mode, a data element representing field_motion_type isdescribed after a data element representing macroblock_type.

The field_motion_type data element is a 2-bit code showing the motionprediction of the macroblock of a field. A field_motion_type value of“01” indicates that there is one prediction vector and the predictiontype is a field-based prediction type. A field_motion_type value of “10”indicates that there is two prediction vectors and the prediction typeis a 18×8 macroblock-based prediction type. A field_motion_type value of“11” indicates that there is one prediction vector and the predictiontype is a dual-prime prediction type.

If the picture is transferred in the frame prediction mode,frame_period_frame_dct indicates that frame_motion_type exists in thebitstream and frame_period_frame_dct also indicates that dct_type existsin the bitstream, a data element representing dct_type is describedafter a data element representing macroblock_type. It should be notedthat the dct_type data element is data used for indicating whether theDCT is carried out in the frame-DCT mode or the field-DCT mode.

As shown in FIG. 61, if the referenced macroblock is either aforward-prediction macroblock or an intra-macroblock completing concealprocessing, a data element defined by a motion_vectors(0) function isdescribed. If the referenced macroblock is a backward-predictionmacroblock, a data element defined by a motion_vectors(1) function isdescribed. It should be noted that the motion_vectors(0) function is afunction used for describing a data element related to a first motionvector and the motion_vectors(1) function is a function used fordescribing a data element related to a second motion vector.

As shown in FIG. 63, the motion_vectors(s) function is a function usedfor describing a data element related to a motion vector.

If there is one motion vector and the dual-prime prediction mode is notused, data elements defined by motion_vertical_field_select[0] [s] andmotion_vector[0] [s] are described.

The motion_vertical_field_select[r] [s] is a flag used for indicatingthat the first vector, be it a forward-prediction or backward-predictionvector, is a vector made by referencing the bottom field or the topfield. The subscript [r] indicates the first or second vector whereasthe subscript [s] indicates a forward-prediction or backward-predictionvector.

As shown in FIG. 64, the motion_vector(r, s) function is a function usedfor describing a data array related to motion_code[r] [s] [t], a dataarray related to motion_residual[r] [s] [t] and data representingdmvector[t].

The motion_code[r] [s] [t] is data with a variable length used forrepresenting the magnitude of a motion vector in terms of a value in therange −16 to +16. motion_residual[r] [s] [t] is data with a variablelength used for representing a residual of a motion vector. Thus, byusing the values of motion_code[r] [s] [t] and motion_residual[r] [s][t], a detailed motion vector can be described. The dmvector[t] is dataused for scaling an existing motion vector with a time distance in orderto generate a motion vector in one of the top and bottom fields (forexample, the top field) in the dual-prime prediction mode and used forcorrection in the vertical direction in order to reflect a shift in thevertical direction between lines of the top and bottom fields. Thesubscript [r] indicates the first or second vector whereas the subscript[s] indicates a forward-prediction or backward-prediction vector. Thesubscript [t] indicates that the motion vector is a component in thevertical or horizontal direction.

First of all, the motion_vector(r, s) function describes a data array torepresent motion_code[r] [s] [0] in the horizontal direction as ahistory stream as shown in FIG. 64. The number of bits of bothmotion_residual[0] [s] [t] and motion_residual[1] [s] [t] is representedby f_code[s] [t]. Thus, a value of f_code[s] [t] other than “1”indicates that motion_residual[r] [s] [t] exists in the bitstream. Thefact that motion_residual[r] [s] [0], a horizontal-direction component,is not “1” and motion_code[r] [s] [0], a horizontal-direction component,is not “0” indicates that a data element representing motion_residual[r][s] [0] is included in the bit stream and the horizontal-directioncomponent of the motion vector exists. In this case, a data elementrepresenting motion_residual[r] [s] [0], a horizontal component, is thusdescribed.

Subsequently, a data array to represent motion_code[r] [s] [1] isdescribed in the vertical direction as a history stream. Likewise, thenumber of bits of both motion_residual[0] [s] [t] and motion_residual[1][s] [t] is represented by f_code[s] [t]. Thus, a value of f_code[s] [t]other than “1” indicates that motion_residual[r] [s] [t] exists in thebitstream. The fact that motion_residual[r] [s] [1], avertical-direction component, is not “1” and motion_code[r] [s] [1], avertical-direction component, is not “0” indicates that a data elementrepresenting motion_residual[r] [s] [1] is included in the bitstream andthe vertical-direction component of the motion vector exists. In thiscase, a data element representing motion_residual[r] [s] [1], a verticalcomponent, is thus described.

It should be noted that, in the variable-length format, the historyinformation can be eliminated in order to reduce the transfer rate oftransmitted bits.

For example, in order to transfer macroblock_type and motion_vectors( ),but not to transfer quantiser_scale_code, slice_quantiser_scale_code isset at “00000” in order to reduce the bit rate.

In addition, in order to transfer only macroblock_type but not totransfer motion_vectors( ), quantiser_scale_code and dct_type, “notcoded” is used as macroblock_type in order to reduce the bit rate.

Furthermore, in order to transfer only picture_coding_type but not totransfer all information following slice( ), picture_data( ) having noslice_start_code is used in order to reduce the bit rate.

As described above, in order to prevent 23 consecutive bits of 0 fromappearing in user_data, a “1” bit is inserted for every 22 bits. Itshould be noted that, however, a “1” bit can also be inserted for eachnumber of bits smaller than 22. In addition, instead of insertion of a“1” bit by counting the number of consecutive 0 bits, a “1” bit can alsobe inserted by examining Byte_allign.

In addition, in the MPEG, generation of 23 consecutive bits of 0 isprohibited. In actuality, however, only a sequence of such 23 bitsstarting from the beginning of a byte is a problem. That is to say, asequence of such 23 bits not starting from the beginning of a byte isnot a problem. Thus, a “1” bit may be inserted for each typically 24bits at a position other than the LSB.

Furthermore, while the history information is made in a format close toa video elementary stream as described above, the history informationcan also be made in a format close to a packetized elementary stream ora transport stream. In addition, even though user_data of the ElementaryStream is placed in front of picture_data according to the abovedescription, user_data can be placed at another location as well.

It should be noted that programs to be executed by a computer forcarrying out pieces of processing described above can be presented tothe user through network presentation media such as the Internet or adigital satellite in addition to presentation media implemented by aninformation recording medium such as a magnetic disc and a CD-ROM.

What is claimed is:
 1. Decoding apparatus for decoding an encodedstream, comprising: receiving means for receiving (a) the encoded streamwhich includes a current quantization scale calculated upon productionof each data block of the encoded stream, and (b) history quantizationscales utilized in more than two past encoding or decoding processes foreach data block of the encoded stream; decoding means for decoding thereceived encoded stream utilizing the received current quantizationscale to produce image data; and output means for outputting the historyquantization scales, the current quantization scale and the image dataproduced by said decoding means for each data block so that, when theimage data produced by said decoding means is re-encoded by a succeedingencoder, a largest quantization scale is selected by the succeedingencoder from the received history and current quantization scales as autilization scale for re-encoding the image data, wherein the historyand current quantization scales are included in the encoded stream, andsaid receiving means acquires the history and current quantizationscales from the encoded stream.
 2. The decoding apparatus according toclaim 1, wherein said output means multiplexes the received history andcurrent quantization scales with the image data produced by saiddecoding means.
 3. The decoding apparatus according to claim 1, whereinthe history quantization scales and the current quantization scale areincluded in different areas of the encoded stream.
 4. The decodingapparatus according to claim 1, wherein the encoded stream is encoded inaccordance with an MPEG standard which has a sequence layer, a GOPlayer, a picture layer, a slice layer, and a macro block layer.
 5. Thedecoding apparatus according to claim 1, wherein the encoded streamrepresents pictures all of which are encoded as I pictures.
 6. Adecoding method for decoding an encoded stream, comprising steps of:receiving (a) the encoded stream which includes as current quantizationscale calculated upon production of each data block of the encodedstream, and (b) history quantization scales utilized in more than twopast encoding or decoding processes for each data block of the encodedstream; decoding the received encoded stream utilizing the receivedcurrent quantization scale to produce image data; and outputting thehistory quantization scales, the current quantization scale and theimage data for each data block so that, when the decoded image data isto be re-encoded by a succeeding encoder, a largest quantization scaleis selected by the succeeding encoder from the received history andcurrent quantization scales to be utilized as a utilization quantizationscale, wherein the history and current quantization scales are includedin the encoded stream, and said receiving step acquires the history andcurrent quantization scales from the encoded stream.
 7. Decodingapparatus for decoding an encoded stream, comprising: receiving meansfor receiving (a) the encoded stream and (b) history quantization scalesutilized in more than two past encoding or decoding for each data blockof the encoded stream; decoding means for decoding the received encodedstream to produce image data; and output means for outputting thehistory quantization scales and the decoded image data for each datablock so that, when the produced image data is re-encoded by asucceeding encoder, a largest quantization scale is selected by thesucceeding encoder from the received history quantization scales as autilization quantization scale for re-encoding the image data, whereinthe history and current quantization scales are included in the encodedstream, and said receiving means acquires the history and currentquantization scales from the encoded stream.
 8. A decoding method fordecoding an encoded stream, comprising the steps of: receiving (a) theencoded stream and (b) history quantization scales utilized in more thantwo past encoding or decoding of each data block of the encoded stream;decoding the received encoded stream to produce image data; andoutputting the history quantization scales and the decoded image datafor each data block so that, when the produced image data is re-encodedby a succeeding encoder, a largest quantization scale is selected by thesucceeding encoder from the received history quantization scales, as autilization quantization scale for re-encoding the image data, whereinthe history and current quantization scales are included in the encodedstream, and said receiving step acquires the history and currentquantization scales from the encoded stream.
 9. Decoding apparatus fordecoding an encoded stream, comprising: a receiver for receiving (a) theencoded stream which includes a current quantization scale calculatedupon production of each data block of the encoded stream, and (b)history quantization scales utilized in more than two past encoding ordecoding processes for each data block of the encoded stream; a decoderfor decoding the received encoded stream utilizing the received currentquantization scale to produce image data; and an output for outputtingthe history quantization scales, the current quantization scale and theimage data produced by said decoder for each data block so that, whenthe image data produced by said decoder is re-encoded by a succeedingencoder, a largest quantization scale is selected by the succeedingencoder from the received history and current quantization scales as autilization quantization scale for re-encoding the image data, whereinthe history and current quantization scales are included in the encodedstream, and said receiver acquires the history and current quantizationscales from the encoded stream.
 10. Decoding apparatus for decoding anencoded stream, comprising; a receiver for receiving (a) the encodedstream and (b) history quantization scales utilized in more than twopast encoding or decoding for each data block of the encoded stream; adecoder for decoding the received encoded stream to produce image data;and an output for outputting the history quantization scales and thedecoded image data for each data block so that, when the produced imagedata is re-encoded by a succeeding encoder, a largest quantization scaleis selected by the succeeding encoder from the received historyquantization scales as a utilization quantization scale for re-encodingthe image data, wherein the history and current quantization scales areincluded in the encoded stream, and said receiver acquires the historyand current quantization scales from the encoded stream. 11.Transmission apparatus, comprising: receiving means for receiving (a) anencoded stream to be decoded to produce image data, the encoded streamincludes a current quantization scale calculated upon production of eachdata block of the encoded stream, and (b) history quantization scalesutilized in more than two past encoding or decoding processes for eachdata block of the encoded stream; and transmission means fortransmitting the history quantization scales and the currentquantization scale for each data block so that, when the image data isre-encoded by a succeeding encoder, a largest quantization scale isselected by the succeeding encoder from the history and currentquantization scales, as a utilization quantization scale for re-encodingthe image data, wherein the history and current quantization scales areincluded in the encoded stream, and said receiving means acquires thehistory and current quantization scales from the encoded stream.
 12. Atransmission method for transmitting a quantization scale, comprisingthe steps of: receiving, (a) an encoded stream to be decoded to produceimage data, including a current quantization scale calculated uponproduction of each data block of the encoded stream, and (b) historyquantization scales utilized in more than two past encoding and decodingprocesses for each data block of the encoded stream; and transmittingthe history quantization scales and the current quantization scale foreach data block so that, when the image data is re-encoded by asucceeding encoder, a largest quantization scale is selected by thesucceeding encoder from the history and current quantization scales, asa utilization quantization scales for re-encoding the image data,wherein the history and current quantization scales are included in theencoded stream, and said receiving step acquires the history and currentquantization scales from the encoded stream.
 13. A transmissionapparatus, comprising: receiving means for receiving (a) an encodedstream to be decoded to produce image data, and (b) history quantizationscales utilized in more than two past encoding or decoding for each datablock of the encoded stream; and transmission means for transmitting thehistory quantization scales for each data block so that, when the imagedata is re-encoded by a succeeding encoder, a largest quantization scaleis selected by the succeeding encoder from the history quantizationscales as a utilization quantization for re-encoding of the image data,wherein the history and current quantization scales are included in theencoded stream, and said receiving means acquires the history andcurrent quantization scales from the encoded stream.
 14. A transmissionmethod, comprising the steps of: receiving (a) an encoded stream to bedecoded to produce image data, and (b) history quantization scalesutilized in more than two past encoding or decoding for each data blockof the encoded stream; and transmitting the history quantization scalesfor each data block so that, when the image data is re-encoded by asucceeding encoder, a largest quantization scale is selected by thesucceeding encoder from the history quantization scales as a utilizationquantization scale for re-encoding of the image data, wherein thehistory and current quantization scales are included in the encodedstream, and said receiving step acquires the history and currentquantization scales from the encoded stream.
 15. Transmission apparatus,comprising: a receiver for receiving (a) an encoded stream to be decodedto produce image data, the encoded stream including a currentquantization scale calculated upon production of each data block of theencoded stream and (b) history quantization scales utilized in more thantwo past encoding or decoding processes for each data block of theencoded stream; and a transmitter for transmitting the historyquantization scales and the current quantization scale for each datablock so that, when the image data is re-encoded by a succeedingencoder, a largest quantization scale is selected by the succeedingencoder from the history and current quantization scales, as autilization quantization scale for re-encoding the image data, whereinthe history and current quantization scales are included in the encodedstream, and said receiver acquires the history and current quantizationscales from the encoded stream.
 16. A transmission apparatus,comprising; a receiver for (a) receiving an encoded stream to be decodedto produce image data, and (b) history quantization scales utilized inmore than two past encoding or decoding for each data block of theencoded stream; and a transmitter means for transmitting the historyquantization scales for each data block so that, when the image data isre-encoded by a succeeding encoder, a largest quantization scale isselected by the succeeding encoder from the history quantization scalesas a utilization quantization scale for re-encoding of the image data,wherein the history and current quantization scales are included in theencoded stream, and said receiver acquires the history and currentquantization scales from the encoded stream.